Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 17th June 2011, 23:32   #721  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Quote:
Originally Posted by kolak View Post
I use one of Canopus Lossless/HQX/HQ (or uncompressed).

I have to try again 64bit, but last time I had no luck. Couldn't make it working at all

Andrew
http://forum.doom9.org/showpost.php?...7&postcount=17
__________________
my repositories
Chikuzen is offline  
Old 18th June 2011, 20:16   #722  |  Link
smooth_hands
Registered User
 
Join Date: May 2010
Posts: 4
Quote:
Originally Posted by -Vit- View Post
Well, you can safely use a faster preset with HD material, e.g.
QTGMC( "Faster" )


Also, you should usually output to a lossless intermediate file when using QTGMC on HD. Then encode that lossless file to x264 or whatever. Otherwise Avisynth will run out of memory (it's still a 32-bit app, so limited to 2Gb).
Vit, would you mind sharing the methods/command line calls you use for the lossless intermediate file step? By looking at the command line parameters in MeGUI, I have figured out how to use mencoder to do this, but the intermediate files it produces are massive (like 250+GB for a normal length movie if I recall). Is this unavoidable?
smooth_hands is offline  
Old 18th June 2011, 21:03   #723  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
In MeGUI just check "Add pre-rendering job". That will first run the avisynth script to output a lossless huffyuv file, then encode that file in a second pass. The intermediate file will be around 200Gb per hour at HD. It will be deleted after processing if you have "Delete intermediate files" checked in Options.

If you don't have the temporary space:
- Don't use a pre-rendering job - reduce settings/threads so it can cope
- Use avs2yuv (or avs2pipe?) to pipe the avisynth output to a separate x264 process to do the encoding. Each process will get it's own memory space. I haven't tried this, I saw the idea here, though I'm sure others use this method for the same reasons.

Last edited by -Vit-; 18th June 2011 at 21:06.
-Vit- is offline  
Old 19th June 2011, 15:05   #724  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: Poland
Posts: 2,471
Quote:
Originally Posted by Chikuzen View Post
Yes- this is some solution, but if you have few files, than it takes a bit of additional work. If we could make some batch file, which would do all automatically, than that would be more useful

I'll try and check how much my PC can do and how much faster it's compared to single thread.

I have 24GB of RAM, so this should not be a problem.

Andrew
kolak is offline  
Old 19th June 2011, 19:46   #725  |  Link
smooth_hands
Registered User
 
Join Date: May 2010
Posts: 4
Quote:
Originally Posted by -Vit- View Post
In MeGUI just check "Add pre-rendering job". That will first run the avisynth script to output a lossless huffyuv file, then encode that file in a second pass. The intermediate file will be around 200Gb per hour at HD. It will be deleted after processing if you have "Delete intermediate files" checked in Options.

If you don't have the temporary space:
- Don't use a pre-rendering job - reduce settings/threads so it can cope
- Use avs2yuv (or avs2pipe?) to pipe the avisynth output to a separate x264 process to do the encoding. Each process will get it's own memory space. I haven't tried this, I saw the idea here, though I'm sure others use this method for the same reasons.
Thanks for the response. The MeGUI option is the same as what I was describing; I just take the same parameters and enter them at the command line. I want to not bring MeGUI into the equation because I have a batch file I use to do all my processing, and MeGUI adds a non-automated step right in the middle of the chain.

I have tried avs2yuv but it always seemed to go slower AND be less stable for me. I also tried reducing settings as per your guide but making it through anything longer than about 15-30min (real-time) of HD video is a crapshoot.

I just wanted to make sure there wasn't some alternative that was as stable as, but faster than, simply running QTGMC in non-multithreaded mode. I don't seem to ever get crashes that way, and it's quicker than doing 2 passes (lossless via mencoder->lossy via x264), but obviously significantly slower than 1 pass running QTGMC in multithreaded mode.

How much memory should one have to have a good chance of avoiding these crashes, anyway? I have 4GB (Windows 7 x64) with no other programs running besides basically bare minimum Windows processes and Microsoft Security Essentials.

I just love this filter so much for interlaced content that I want to do everything in my power to get it into my workflow whenever feasible
smooth_hands is offline  
Old 19th June 2011, 23:37   #726  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
You need 2Gb spare memory for 32-bit avisynth. 4Gb and a low system load sounds OK.

The alternative posted above, splitting an encode into chunks, is the other way to get more speed whilst retaining stability. It's not the speedup of SetMTMode but it can help. However, to take best advantage you may need more memory - that approach relies on creating several processes rather than several threads. Each process can have its own 2Gb.

One other thing: mencoder was dropped from MeGUI quite a while ago, in favor of ffmpeg for lossless output. I use the dev builds, dunno if that change has made it to the stable builds yet.
-Vit- is offline  
Old 23rd June 2011, 16:30   #727  |  Link
dfan
Registered User
 
Join Date: May 2011
Posts: 6
hi
how i could make a timecodes' file correctly with tdecimate?

if i use qtgmc+selecteven+tdecimate(+ovr) then there are some zones to 24fps but that really are to 30fps (tfm+tdecimate+ovr detect them correctly ._.). Any idea for fix this?

thx
dfan is offline  
Old 26th June 2011, 16:58   #728  |  Link
nhope
partially-informed layman
 
Join Date: Jan 2002
Location: Bangkok, Thailand
Posts: 314
PAL to NTSC

I've started testing the following script for PAL DV > NTSC-for-DVD conversion, as an improvement over my previous method of blending frames in Vegas Pro. Initial results are promising, although of course it's slow. I would welcome any comments or suggestions for improvement. Thanks

Code:
#Frameserve from Vegas Pro in YUY2.

AviSource("d:\fs.avi")
ColorYUV(levels="TV->PC") #Reverses frameserving levels squeeze
QTGMC()
Spline36Resize(720,480)

#Could save lossless intermediate here

super = MSuper(pel=2)
backward_vec = MAnalyse(super, isb = true)
forward_vec = MAnalyse(super, isb = false)
MFlowFps(super, backward_vec, forward_vec, num=60000, den=1001)

AssumeBFF()
SeparateFields()
SelectEvery(4,0,3)
weave()
nhope is offline  
Old 26th June 2011, 18:35   #729  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,343
Nick,

Is your source interlaced? If so, does this script give correct results?

[edit] Forget this post. I didn't see the QTGMC line.

Last edited by johnmeyer; 26th June 2011 at 18:36. Reason: Realized QTGMC was used
johnmeyer is online now  
Old 26th June 2011, 20:02   #730  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Can use QTGMC's own motion vectors, which it exposes for later use. This will be faster and likely give better vectors. But then you must resize after the flow
Code:
#...
QTGMC( SubPel=2 ) # SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) # Only need multiple levels for MAnalyse
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=60000, den=1001)
Spline36Resize(720,480)
#...
[Edit: If you use faster presets on QTGMC then you will need to set hpad/vpad to 16 on the MSuper line]

This process is detailed in the QTGMC docs under the heading "Accessing QTGMC Motion Vectors"

Or you could try Interframe to do the 50->60fps. The script is still essentially MFlowFPS, but it has been tweaked heavily.

Last edited by -Vit-; 26th June 2011 at 20:39.
-Vit- is offline  
Old 26th June 2011, 21:57   #731  |  Link
nhope
partially-informed layman
 
Join Date: Jan 2002
Location: Bangkok, Thailand
Posts: 314
Thanks very much -Vit-. I hadn't realised that Interframe could do framerate conversions other than doubling.

So now I want to try your 2 suggestions; The MFlowFps script using QTGMC vectors and the following script using InterFrame:

Code:
#Frameserve from Vegas Pro in YUY2

AviSource("d:\fs.avi")
ColorYUV(levels="TV->PC") #Reverses frameserving levels squeeze
QTGMC()
Spline36Resize(720,480)

InterFrame(FlowPath="C:\Program Files\AviSynth 2.5\plugins\")

AssumeBFF()
SeparateFields()
SelectEvery(4,0,3)
weave()
One of the Interframe dependencies is a modified mvtools2.dll from here. How would you recommend managing 2 different versions of mvtools2.dll (one for QTGMC and one for InterFrame)? Or could the same version be used for both? Would I have to split the script and load plugins from different folders?

Also, RemoveGrainSSE3.dll is in the InterFrame dependencies folder, but I'm running RemoveGrainSSE2.dll with QTGMC. I guess I should go for one or the other, but which?
nhope is offline  
Old 27th June 2011, 15:38   #732  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
I skimmed the SVP modded MVTools code a while back and the changes were fairly limited in scope, so I imagine it will still work for normal use [that's just a guess though, I don't actually use it myself]. Just note it's based on 2.5.11.1. The latest version is 2.5.11.2, which fixes a crash bug that affected QTGMC with certain resolutions/blocksizes. But that shouldn't affect your use case though.

Use RemoveGrainSSE2. There was a bug in RemoveGrainSSE3 until the final version, but that one has some changed and undocumented functionality so it shouldn't be relied on. There is no noticeable difference between the two anyway.
-Vit- is offline  
Old 27th June 2011, 23:56   #733  |  Link
SubJunk
Registered User
 
Join Date: Jun 2010
Posts: 441
Quote:
Originally Posted by nhope View Post
How would you recommend managing 2 different versions of mvtools2.dll (one for QTGMC and one for InterFrame)? Or could the same version be used for both?
Yeah I don't think you will have any problems using the same one for both. Most of the SVP changes have been done to a new function they created called MSmoothFPS, so MFlowBlur and MSuper (which QTGMC uses) still work as they did before AFAIK.

Quote:
Originally Posted by nhope View Post
Also, RemoveGrainSSE3.dll is in the InterFrame dependencies folder, but I'm running RemoveGrainSSE2.dll with QTGMC. I guess I should go for one or the other, but which?
The one with the highest number But seriously it won't make much difference, you can use either one
SubJunk is offline  
Old 28th June 2011, 12:01   #734  |  Link
nhope
partially-informed layman
 
Join Date: Jan 2002
Location: Bangkok, Thailand
Posts: 314
Thanks -Vit- and SubJunk. I ran the InterFrame script with the original RemoveGrainSSE2.dll and mvtools2.dll supplied in -Vit-'s required plugins pack and I had no problems.

I found little difference in the quality of the result, but in my single-threaded environment the MFlowFps method was about 7-8% faster than the InterFrame method. Also InterFrame doesn't support YUY2, and I prefer not to use YV12 in this case because it gives me this chroma shift error when I take interlaced YV12 into CCE Basic for MPEG-2 encoding.

So anyway, the following script gives a stunningly good PAL>NTSC conversion compared to anything I've tried before (which is A LOT of software methods). So good in fact that I think I can start just offering NTSC DVDs instead of both PAL and NTSC depending on territory (my original footage is PAL, which won't play worldwide). Thank you!

Code:
#Frameserve from Vegas Pro in YUY2

AviSource("d:\fs.avi")
ColorYUV(levels="TV->PC") #Reverses frameserving levels squeeze

QTGMC( SubPel=2 ) # SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) # Only need multiple levels for MAnalyse
#super = MSuper(levels=1, pel=2, hpad=16, vpad=16) # For faster QTGMC presets
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=60000, den=1001)
Spline36Resize(720,480)

AssumeBFF()
SeparateFields()
SelectEvery(4,0,3)
weave()
Just one thing. There are still jaggies in evidence such as around the diver's mask in this screengrab. I suspect they are unavoidable, but any suggestions to reduce them?

nhope is offline  
Old 28th June 2011, 12:37   #735  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Quote:
Originally Posted by nhope View Post
There are still jaggies in evidence such as around the diver's mask in this screengrab. I suspect they are unavoidable, but any suggestions to reduce them?
That can't be judged from one screenshot, because you're re-interlacing. Deactivate the Separate/Select/Weave sequence, and check if there still are jaggies then.

(Though, straight from the guts, those jaggies don't really look like they're re-interlacing related, but rather like something's going seriously wrong. It pretty much looks like aliasing from a simple standard deinterlacer!)


Oh, BTW ... if the frameserved output really has a PC-->TV levels squeeze, then that's not good at all ... in particular for underwater shots. (During level squeeze information is getting lost, and you won't get that loss back by expanding again....)
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 28th June 2011 at 12:42.
Didée is offline  
Old 28th June 2011, 14:54   #736  |  Link
nhope
partially-informed layman
 
Join Date: Jan 2002
Location: Bangkok, Thailand
Posts: 314
Quote:
Originally Posted by Didée View Post
That can't be judged from one screenshot, because you're re-interlacing. Deactivate the Separate/Select/Weave sequence, and check if there still are jaggies then.
Sorry. that screenshot was misleading. It was actually of a single field doubled in height after re-interlacing ... I did as you suggested and the pre-re-interlaced 59.94p frames are as smooth as a baby's bottom. The jaggies/combing were showing up on my 5-year-old LG DVD player + Samsung CRT TV. Perhaps it's normal and I'm looking too hard. Or perhaps, even though they are "multi-system", they are from PAL-land and don't behave the same as a TV from NTSC-land. It's definitely no show-stopper anyway.

Quote:
Oh, BTW ... if the frameserved output really has a PC-->TV levels squeeze, then that's not good at all ... in particular for underwater shots. (During level squeeze information is getting lost, and you won't get that loss back by expanding again....)
Vegas works internally in RGB. The Debugmode frameserver will serve RGB or YUY2. When frameserved out in RGB there is no levels squeeze. When the frameserver does an RGB>YUY2 conversion it uses Rec601 and so I need to expand the levels out again in AviSynth. Perhaps I'd be better serving in RGB and converting to YUY2 with matrix="PC.601" in AviSynth?
nhope is offline  
Old 30th June 2011, 19:17   #737  |  Link
nhope
partially-informed layman
 
Join Date: Jan 2002
Location: Bangkok, Thailand
Posts: 314
This is where I have ended up and I'm really happy with the results, but didn't test the NTSC>PAL script much since I don't have original SD NTSC footage.

(Note that Vegas works internally in RGB, so I'm using the path of least conversions)

PAL to NTSC

Code:
#Frameserve PAL source from Vegas in RGB24 format

AviSource("d:\fs.avi")
AssumeBFF()
ConvertToYUY2(interlaced=true, matrix="PC.601")

QTGMC( SubPel=2 ) #SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) #Only need multiple levels for MAnalyse
#super = MSuper(levels=1, pel=2, hpad=16, vpad=16) # For very fast and quicker QTGMC presets
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=60000, den=1001)
Spline36Resize(720,480)

SeparateFields()
SelectEvery(4,0,3)
weave()
NTSC to PAL

Code:
#Frameserve NTSC source from Vegas in RGB24 format

AviSource("d:\fs.avi")
AssumeBFF()
ConvertToYUY2(interlaced=true, matrix="PC.601")

QTGMC( SubPel=2 ) # SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) # Only need multiple levels for MAnalyse
#super = MSuper(levels=1, pel=2, hpad=16, vpad=16) # For very fast and quicker QTGMC presets
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=50, den=1)
Spline36Resize(720,576)

SeparateFields()
SelectEvery(4,0,3)
weave()
1080-50i to SD NTSC

Code:
#Frameserve 1080-50i source from Vegas in RGB24 format

AviSource("d:\fs.avi")
AssumeTFF()
ConvertToYUY2(interlaced=true, matrix="PC.601")

Spline36Resize(720,height)
QTGMC("faster", sharpness=0, SubPel=2) #SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) # Only need multiple levels for MAnalyse
#super = MSuper(levels=1, pel=2, hpad=16, vpad=16) # For very fast and quicker QTGMC presets
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=60000, den=1001)
Spline36Resize(width,480)

blur(0.0,1.0)
sharpen(0.0,0.75) #Adjust 2nd value between 0.5 and 1.0 to taste

SeparateFields()
SelectEvery(4,0,3)
Weave()
One question. For which exact QTGMC presets would hpad=16 and vpad=16 be required in these 3 cases?

Last edited by nhope; 1st July 2011 at 05:19.
nhope is offline  
Old 30th June 2011, 21:28   #738  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by nhope View Post
One question. For which exact QTGMC presets would hpad=16 and vpad=16 be required in these 3 cases?
"Very Fast" and quicker presets for all those 3 scripts.

This v/hpad requirement is only there because QTGMC still contains a workaround for a bug in MVTools 2.5.11.1 and earlier, fixed in the latest version 2.5.11.2. I kept the workaround in case people didn't upgrade their MVTools. The next QTGMC version will drop the workaround and will use the default v/hpad in general (and will require the latest version of MVTools).
-Vit- is offline  
Old 1st July 2011, 05:18   #739  |  Link
kypec
User of free A/V tools
 
kypec's Avatar
 
Join Date: Jul 2006
Location: SK
Posts: 830
Quote:
Originally Posted by -Vit- View Post
The next QTGMC version will drop the workaround and will use the default v/hpad in general (and will require the latest version of MVTools).
Mind to tell what new exciting improvements can we look forward to in this next version?
kypec is offline  
Old 1st July 2011, 08:33   #740  |  Link
aegisofrime
Registered User
 
Join Date: Apr 2009
Posts: 462
Quote:
Originally Posted by kypec View Post
Mind to tell what new exciting improvements can we look forward to in this next version?
I would like to know that as well.

Still, if the next version of QTGMC requires the latest MVTools, I do hope somebody can compile a 64-bit version of that. 64-bit has consistently been more stable and faster than 32-bit for me, although the newest version of NNEDI3 has somewhat closed the gap considerably.

Also, on a slightly OT note, if somebody can post a guide on how we can do our own 64-bit compilations that would be great.
aegisofrime is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.