Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 13th February 2011, 17:07   #281  |  Link
henryho_hk
Registered User
 
Join Date: Mar 2004
Posts: 889
I know inputtype=2 is meant for fixing badly deinterlaced footages. But it will appear to me that the original TGMC is doing a much better job, especially for those ill-brained field-duplicated deinterlacers. What parameters I should be tuning for these sources?
henryho_hk is offline  
Old 13th February 2011, 22:19   #282  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Have you tried InputType=1? That's should be the same as what original TGMC does but without the frame duplication (which interferes with the motion analysis). It can be the appropriate method for less degraded sources.

InputType=2,3 drop half the fields - which is pretty extreme, but necessary when the source is a real mess.

There's also this approach from a previous post:
Code:
s = WhateverSource("Your.src")

# Make versions from upper fields and lower fields (use TR2 and ProgSADMask also if you wish)
t = s.QTGMC( InputType=2, ... )   # These lines work well with "SourceMatch=1, Lossless=2" (need v3.01+)
b = s.QTGMC( InputType=3, ... )

# Now merge the results in some way. Choose ONLY ONE of the lines below and delete the other
Merge( t, b )                 # Smooth result, may need more sharpening
t.Repair( Merge( t, b ), 1 )  # Sharper result, could try 2 at end of this line
I find that latter approach works very well on some sources, less so on others. It's slower than it could be of course...
-Vit- is offline  
Old 13th February 2011, 22:51   #283  |  Link
@MeGui
Registered User
 
Join Date: Apr 2008
Posts: 39
Vit, nice one mate...))
I have i7 core, so can u tell me how to use max of my cores?
I have Avisynth 2.5.8 MT installed.
What would be "lines" only for pure de-interlacing?

Tnx in advance...
@MeGui is offline  
Old 13th February 2011, 23:43   #284  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Straight deinterlacing using multi-threading:
Code:
SetMemoryMax(M) # Optional line. Leave it out at first. See below for value M
SetMTMode(5, X) # See below for value X
WhateverSource("Your.src")
SetMTMode(2)
QTGMC( Preset="Slow", EdiThreads=Y ) # Choose preset based on overall speed/quality you want. See below for value Y
Distributor() # This line may or may not be necessary, try it without - it may be faster
Setting X
- Start at the number of cores in your machine
- If it crashes, decrease 1 at a time
- Otherwise increase 1 at a time until CPU usage is very, very close to 100%, don't go too far or it will slow down

Setting Y
- Start at about half number of cores and tweak upwards or downwards for best speed. I find Y=1 often works best when using SetMTMode.
- Obviously this is a balance with X

Setting M
- Using the SetMemoryMax line and choosing a good value for M can allow you to use more threads and so get more speed. Particularly important for slower settings
- Try 400,600,800,1000 etc.

[I should add this to the OP / docs...]
-Vit- is offline  
Old 14th February 2011, 00:13   #285  |  Link
@MeGui
Registered User
 
Join Date: Apr 2008
Posts: 39
hmmm, using Virtualdub 1.9.11 and no meather what, crashed...
@MeGui is offline  
Old 14th February 2011, 00:19   #286  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
With Virtualdub I don't think you need the Distributor() line.

Try different versions of MT AviSynth if you get stability issues: Standard 2.57MT, SEt's 2.58MT or SEt's 2.6MT

Last edited by -Vit-; 28th April 2011 at 13:51.
-Vit- is offline  
Old 14th February 2011, 00:53   #287  |  Link
@MeGui
Registered User
 
Join Date: Apr 2008
Posts: 39
I think is not Avisynth issue...this "MT" running good...

http://pastebin.com/EAmLksev
@MeGui is offline  
Old 14th February 2011, 01:27   #288  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
I don't know what that paste was about.

But in any case, MT avisynth is buggy. It might work with one script but not another. You should try the other versions. SEt's versions appear to be most reliable, but each system is different. There are many posts on the subject in this thread.
-Vit- is offline  
Old 14th February 2011, 01:28   #289  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,121
Quote:
Originally Posted by @MeGui View Post
hmmm, using Virtualdub 1.9.11 and no meather what, crashed...
I have constant crashes also- my source is 1920x1080 50i and new QTGMC is almost unusable with MT.

Works fine when using simple command:
QTGMC(Preset="Fast")- any setting above crashes.


Andrew
kolak is offline  
Old 14th February 2011, 01:52   #290  |  Link
@MeGui
Registered User
 
Join Date: Apr 2008
Posts: 39
Yes, indeed ...but, i can still get only 1.14 rendering...not more
@MeGui is offline  
Old 14th February 2011, 02:44   #291  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by kolak View Post
New QTGMC is almost unusable with MT
I would be interested to know which version of QTGMC you last had stable...

I've made very few changes to the core algorithm for several versions. There's nothing new in version 3 - the presets on their own should result in the pretty much same filter graph as always. I have been adding extra features, but they're all off by default so it should run just as before when you just provide a basic preset. However, I used to have more explicit conditions to remove unused filters, I now rely on avisynth to do that job when it builds the filter graph - I wonder if that is causing a problem. Maybe I should revert to doing things manually again...
____

EDIT: A quick look at the avisynth parsing code and I think if you don't explicitly put a conditional around an unused filter then its constructor is called anyway [correct me if I'm wrong devs]. That's unfortunate, it could cause unnecessary memory use at the least.

In practical scripting terms it appears to mean that this:
Code:
input = last
processed = (setting == 1) ? input.ComplexFilter() : input
return processed
is better than this:
Code:
input = last
processed = input.ComplexFilter()
return (setting == 1) ? processed : input
These two scripts are equivalent. If setting = 2 then the ComplexFilter plugin isn't going to get used. However, in the second example it will be constructed when the script is parsed at the beginning - this will use up memory unnecessarily. The first example guards it with a condition, the plugin will neither be constructed nor called. [At least, that's what I surmise from a quick glance over the avisynth source code]. I had assumed the second method was OK as I had seen it in a number of other scripts and examples.

I'll revert back to my original method and see if there's a stability difference... stay tuned...

Last edited by -Vit-; 14th February 2011 at 06:02.
-Vit- is offline  
Old 14th February 2011, 10:11   #292  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,373
Quote:
Originally Posted by -Vit- View Post
A quick look at the avisynth parsing code and I think if you don't explicitly put a conditional around an unused filter then its constructor is called anyway [correct me if I'm wrong devs]. That's unfortunate, it could cause unnecessary memory use at the least.
That's correct. And with multi-threaded Avisynth, you will get an instance for each thread, using more memory.

How much impact this has will of course depend on how much memory the filter allocates on construction - many simple filters have nothing beyond a few simple variables, but something that keeps its own frame buffer (for example) will obviously use more.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline  
Old 14th February 2011, 15:59   #293  |  Link
sadie
Registered User
 
Join Date: Sep 2006
Posts: 55
...and I thought I was out of here. Well, I managed to reproduce my bug/error albeit in a still not completely conclusive way. And I'm also at the stage where in passing I'd like to apply a couple of old human adages to this technology: A: Honeymoons never last B: Life is a crap shoot.

That said, and after two successive crashes I managed to get a successful 9 hr encode (fast preset) for my l hr 10 min video. I also managed to custom start three threads for my modest core duo T8100. That should have given it some elbow room. Again, the encode and subsequent tests were avisynthproxy in avidemux and an x264 medium quality preset.

On playback without even looking too hard I noticed some soft softs which are discernable in the following captures which reflect the order with which I tested various things. Areas to look for are bldg to far right, lst floor window 2nd from left and 2nd fl window lst on left. (1) shows the smoothing/obliteration equal to my full encode. I re-reproduced it using different crt and abr settings with identical results. (2) here, I rebooted avisynthproxy in SourceMatch 2 25p mode. The anomaly disappeared. (3) further reboot back into default QTGMC and here the smoothing went bananas, equivalent to what I experienced on Saturday. (4) booted avs script with default QTGMC in 25p (selecteven) mode...the problem disappeared. (5) finally back to original avs in 50p and once again the error was gone. I should add that I think the source sample is not virginal. It might have undergone some slight slow motion filtering.

Vit, you mentioned 'memory corruption'. Whatever that's supposed to mean it doesn't get me very far. From my amateurish experience with computers over the years, in a case like this I'd always thought a hardware problem generally either translate into something that works or breaks. Be it the hardware itself or the task it was trying achieve. But it doesn't 'bend'. And this random smoothing strikes me as being just that: bending.

The problem isn't systematic except for the fact it's only ever occurred in 50p mode, never with 25p. Is there something to be learned here? Maybe I should play it safe and stick with that. (SM=2 by the way was a decided upgrade from SM=1). Further, is there any reason NOT to do a quality archive copy @7000-8000 kbs using sourcematch just in 25p and either a) count on my Sony flatscreen's 100mhz 'motion flow' to iron out the judder or b) later on down the line when I'd like to play around with some 720p upscaling go about doubling the frames then and hopefully not suffer a quality hit. As I always back up the finished DV render on mini-DV tapes that shouldn't be a such a huge risk.

You say, Vit, you've never had a 'failure' in hundreds of cases of QTGMC. But maybe you're on systems using quad-cores or far better. Which brings up the 'minimal configuration' issue. Is there one for this avisynth plugin?

I just had a last thought as I write these words. All the tests I've done were through to an x264 encode. Wouldn't the truly rigorous method be to output to another DV avi file? How much might the encoder end of these operations be interfering in performance/image quality? As I recall x264 loop filter settings bear on perceived sharpness. Doesn't this complicate matters? Thanks for your continuing help.
sadie is offline  
Old 14th February 2011, 16:04   #294  |  Link
sadie
Registered User
 
Join Date: Sep 2006
Posts: 55
Oops, my attachments disappeared.

EDIT: Sorry for error. I'd thought attachments would remain in a link form with info file name. Didn't want to bloat the thread. I've removed pics 4 & 5. The middle image represents what the correct encode should like anyway.
Attached Images
   

Last edited by sadie; 14th February 2011 at 20:26.
sadie is offline  
Old 14th February 2011, 19:20   #295  |  Link
marcopolo80
Registered User
 
Join Date: Feb 2003
Posts: 13
For this kind of source file (known as 1080-60i):

Video
ID : 2064 (0x810)
Menu ID : 100 (0x64)
Format : MPEG Video
Format_Commercial_IfAny : HDV 1080i
Format version : Version 2
Format profile : Main@High 1440
Format settings, BVOP : Yes
Format settings, Matrix : Default
Format settings, GOP : M=3, N=15
Codec ID : 2
Bit rate mode : Constant
Bit rate : 25.0 Mbps
Width : 1 440 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 29.970 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.536

and thinking that almost all the time the camera will shoot in natural and good light-so the footage will look OK-what is the best: to stay interlaced and encode to x264(or MainConcept h264 AVC Broadcast) at 8-12 Mbps(not sure if 12 is not a waste of space) or to go progressive(720p) also encoded in x264 or MC-264avc at 6-8Mbps?
For the second choice which settings for fast encoding speed?And keep 59.94 or to make a pulldown to 23,97?
My script looks now (obvious with the help of some people from this forum):

Code:
SetMemoryMax(900)
SetMTMode(5,6)
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
MPEG2Source("C:\test5min.d2v")
SetMTMode(2,6)
AssumeTFF()
ConvertToYV12(interlaced=true)
Import("C:\Program Files\AviSynth 2.5\plugins\QTGMC3.05.avsi")
spline36resize(1280,720)
QTGMC( Preset="Fast", EdiThreads=4 )
The CPU is AMD x6 1055T@3.3Ghz with 4GB DDR3 1366 and x86 OS

Thanks

Last edited by marcopolo80; 14th February 2011 at 19:29.
marcopolo80 is offline  
Old 14th February 2011, 21:26   #296  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Here's a revised version of QTGMC (v3.10) which reverts to the approach of v2.x. This will call far fewer filter constructors, using less memory and maybe prevent the crashes some people have had. If you couldn't run the recent versions, please give this one a quick test and report back. I haven't updated the OP yet in case this doesn't help. I tested this multi-threaded in VirtualDub (1.9.10) on some 1920x1080x60i material at preset "Slow" - it was fine:

QTGMC v3.10

____

sadie: I've never seen smearing like that out of QTGMC. Might it be x264? That can cause smearing, especially given that it's a fairly complex scene and you said you were going down to CRF 26? And it's strange that it is not consistently reproducible. These are deterministic algorithms, there isn't scope for variation. x264 does have a non-deterministic multi-threaded mode (the order of the threading calls is unpredictable and they get some tiny speed advantages by not worrying about that), but I don't think that mode is a default.

However, x264 is getting in the way with these examples. I always encode lossless 1st pass, then x264 the result. You should try to reproduce the error in a lossless format. Better still you should post that source snippet, so I can try to reproduce the issue.

____

marcopolo80: That seems like a decent start, but you've made a mistake with the resizing - you shouldn't vertically resize before the deinterlacing. The last part of the script should be this:
Code:
#...
spline36resize(1280,1080)
QTGMC( Preset="Fast", EdiThreads=4 )
spline36resize(1280,720)
You should tweak the threading settings to get best speed. About your other questions - there is no "best" choice, you must try each of your options and decide for yourself.

Last edited by -Vit-; 14th February 2011 at 21:43.
-Vit- is offline  
Old 14th February 2011, 22:15   #297  |  Link
marcopolo80
Registered User
 
Join Date: Feb 2003
Posts: 13
Thanks,I'll try those settings.
marcopolo80 is offline  
Old 15th February 2011, 12:32   #298  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 846
-Vit-!
Amazing!!!
Now my HD source work SetMTMode(2,4) and placebo preset without crash (before work only SetMTMode(2,2)). I am also note speedup 20...40%.
Waiting from Stephen R. Savage YUY2 release.
Good work.
yup is offline  
Old 15th February 2011, 14:37   #299  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Thanks yup. I'll take that as confirmation that I've at least fixed something. I've updated the OP to v3.10

Avisynth only calls what it needs when frame-serving, but it initializes everything that isn't explicitly guarded with a condition. QTGMC contains a huge number of filters, many of which are not used unless you choose a particular setting. However, almost all of them were being initialized, which must have been using a great deal of memory - it was hard to tell because the avisynth cache uses a huge amount of memory anyway. Other scripters with conditional / optional processing should note the example above.
-Vit- is offline  
Old 15th February 2011, 15:40   #300  |  Link
@MeGui
Registered User
 
Join Date: Apr 2008
Posts: 39
In My case, calling "mt_masktools-26.dll" in script, did the trick...

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-26.dll")
SetMemoryMax(1024)
SetMTMode(5,8)
DirectShowSource("C:\Documents and Settings\XyZ\Desktop\Svadba2.avi")
SetMTMode(2)
ConvertToYV12(interlaced=true)
Import("C:\Program Files\AviSynth 2.5\plugins\QTGMC3.10.avsi")
spline36resize(720,576)
QTGMC( Preset="Slow", EdiThreads=8 )
spline36resize(720,432)


I'm getting 34.10 fps with "Slow" preset on i7's

Tnx Vit again )

p.s. Vit, this script, is it correct for 720x576 ( DV-PAL-interlaced 25.000)-->720x432?

Last edited by @MeGui; 15th February 2011 at 16:25.
@MeGui is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:30.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.