Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th May 2015, 15:27   #1  |  Link
8-BaLL
Registered User
 
Join Date: Jun 2012
Posts: 75
QTGMC only works if I use ConvertToYV12 on my 1080i HDTV recorded files, why?

Hey guys,

Ive recorded some 1080i stuff on the TV. The files are AVC in .TS containers.

I want to deinterlace it with QTGMC and endcode at 720p.

Problem is- I get an error message when QTGMCing it "interleaved2planar... error".

If I use "ConvertToYV12", then QTGMC works without any trouble.

Do you know why it is the case?, here is Mediainfo of the source files:

Code:
Video
ID                                       : 511 (0x1FF)
Menu ID                                  : 1000 (0x3E8)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.0
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 3 frames
Format settings, GOP                     : M=4, N=28
Codec ID                                 : 27
Duration                                 : 3mn 28s
Bit rate                                 : 7 973 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 fps
Standard                                 : Component
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.154
Stream size                              : 198 MiB (90%)
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Color range                              : Limited
Is the source not compatible with QTGMC without converting?

Here is how my script now looks:

Code:
DirectShowSource("C:\TVcap.ts", audio=false)

ConvertToYV12(interlaced=true)

qtgmc(preset="very slow")

spline64resize(1280, 720)
this way QTGMC works fine, without ConvertToYV12(interlaced=true) I get that interleaved2planar error message.
8-BaLL is offline   Reply With Quote
Old 9th May 2015, 15:58   #2  |  Link
TheSkiller
Registered User
 
Join Date: Dec 2007
Location: Germany
Posts: 632
DirectShowSource is the culprit. Your source already is YV12 (4:2:0), correct, but DirectShowSource seems to convert it to something else (probably YUY2), and therefore you need to convert to YV12 yet again for QTGMC.

Don't use DirectShowSource.
TheSkiller is offline   Reply With Quote
Old 9th May 2015, 16:23   #3  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
I'm sure I recall a similar problem in another thread..... maybe another forum...... where the OS was Win7 and the DirectShow decoder was whatever codec Microsoft supplies with Windows. For some reason it was decoding as YUY2. The DirectShow fix was to install ffdshow and use the Win7DSFilterTweaker to disable the Microsoft codec. Assuming it's the same problem......

But yeah...... indexing with L-smash or ffms2 might be better than using DirectShowSource. Muxing the TS file as an MKV first might also be helpful.
hello_hello is offline   Reply With Quote
Old 9th May 2015, 16:43   #4  |  Link
8-BaLL
Registered User
 
Join Date: Jun 2012
Posts: 75
Quote:
Originally Posted by TheSkiller View Post
DirectShowSource is the culprit. Your source already is YV12 (4:2:0), correct, but DirectShowSource seems to convert it to something else (probably YUY2), and therefore you need to convert to YV12 yet again for QTGMC.

Don't use DirectShowSource.
Thanks for the input. What alternative should I use for the .TS file? With FFvideosource() ?

edit: Ive tested ffms2 and the problem no longer occurs. Thanks for the tip guys.

But isnt ffms2 known as a little unprecise in handling frames or something like that? Or is ffms2 reliable as it is right now?

Also is it normal that my PC only manages 1,5 fps with this script?

Code:
FFVideoSource("c:\TVcap.ts", fpsnum=25, fpsden=1, threads=1)

qtgmc(preset="very slow")

spline64resize(1280, 720)
My CPU load is only around 50% though, which is core i5 @ 4GHz. A little inefficient the qtgmc...

Last edited by 8-BaLL; 9th May 2015 at 17:17.
8-BaLL is offline   Reply With Quote
Old 9th May 2015, 17:31   #5  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
The reason I suggested remuxing as an MKV first is I've found ffms2 to be pretty reliable with MKVs, but a little less so with TS files. Generally the worst that might happen is ffms2 will output the frame rate as 50fps (because it's interlaced) which can usually be fixed by telling it the correct frame rate to use. ie:

FFVideoSource("E:\video.mkv", cachefile="D:\test.mkv.ffindex", fpsnum=25, fpsden=1, threads=1)

DirectShowSource is the least frame accurate method. It's usually okay when encoding from start to finish but with complicated filtering where frames mightn't be requested in order, it's probably best to avoid it.

If you're using the latest "official" stable ffms2, which is getting a bit old, there's some more recent builds to be found in recent posts here if you happen to have any problems. Or try L-Smash.

PS ffms2 requires the Haali Media Splitter for TS files. https://ffmpegsource.googlecode.com/...-avisynth.html

Last edited by hello_hello; 9th May 2015 at 17:36.
hello_hello is offline   Reply With Quote
Old 9th May 2015, 17:37   #6  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,405
For better performance you can try something like this:
Code:
Source("TVcap.ts")

Spline36Resize(1280,height)
QTGMC()
Spline36Resize(1280, 720)
More than likely the difference will be negligible.
Reel.Deel is offline   Reply With Quote
Old 9th May 2015, 19:08   #7  |  Link
8-BaLL
Registered User
 
Join Date: Jun 2012
Posts: 75
Quote:
Originally Posted by hello_hello View Post
The reason I suggested remuxing as an MKV first is I've found ffms2 to be pretty reliable with MKVs, but a little less so with TS files. Generally the worst that might happen is ffms2 will output the frame rate as 50fps (because it's interlaced) which can usually be fixed by telling it the correct frame rate to use. ie:

FFVideoSource("E:\video.mkv", cachefile="D:\test.mkv.ffindex", fpsnum=25, fpsden=1, threads=1)

DirectShowSource is the least frame accurate method. It's usually okay when encoding from start to finish but with complicated filtering where frames mightn't be requested in order, it's probably best to avoid it.

If you're using the latest "official" stable ffms2, which is getting a bit old, there's some more recent builds to be found in recent posts here if you happen to have any problems. Or try L-Smash.

PS ffms2 requires the Haali Media Splitter for TS files. https://ffmpegsource.googlecode.com/...-avisynth.html
Thanks alot for the tipps, I will try the recent ffms2 builds and also remux files into MKV before encoding.

One more small question though- what value should I select in for "threads" in ffms2? When I select a file, it automatically selectes threads=1, I googled a little and some ppl say its best to have threads=0 or skip this option completely and not select any value at all. What is the best choice for a quad core CPU?

I just encoded a test file and it looks amazing Thanks for all the tipps guys!

Last edited by 8-BaLL; 9th May 2015 at 19:39.
8-BaLL is offline   Reply With Quote
Old 10th May 2015, 01:29   #8  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
I use threads=1 because on a couple of odd occasions I had problems without it (I think the source video was Divx one time). Anyway, it just sets the number of threads to be used for decoding and I'm not sure more threads would necessarily speed things up, given the CPU will also be pretty busy with filtering and encoding etc. I can't say I've compared speeds though. Depending on the decoder being used it mightn't have any effect anyway (if it doesn't support threaded decoding). Threads=0 means the same number of threads as you have CPU cores. The default is threads=-1 (minus one) which is the same as threads=0. Is that what you meant when you said it automatically selects threads=1?

I never rule out using DirectShowSource if other methods fail with "problem" files. Sometimes (at least with ffdshow decoding) I've found it'll keep decoding when other methods fail, but I tend to only use it for simple "start to finish" encoding. For anything else I'd use DirectShowSource to decode to a lossless file and then re-encode from there.

Oh..... and there' an issue with AVIs and ffms2 if you use it's frame rate conversion in a script. If you don't it's fine, but for some reason it tends to drop or duplicate frames now and then when it shouldn't and motion can be "jittery" in places (if a video is 25fps and you use fpsnum=25 and fpsden=1 in a script, in theory it should have no effect). I don't think it's been fixed but it only effects AVIs and I tend to go with "remuxing as MKV" for most file types before encoding anyway. I think using the frame rate conversion in each script is a good idea, maybe even more so for TV captures, because without it if there's any missing/null frames the video gets treated as variable frame rate and it can mess with the audio sync. If you add the frame rate conversion stuff to the script (ffms2 and L-Smash both have the same options) then the output should be the correct constant frame rate and frames will be duplicated as required to account for any "missing" frames. Just thought I'd mention it..... but if you have any audio sync problems or the output frame rate is slightly off, try using the frame rate conversion options. If the total frame count changes when you add the appropriate frame rate "conversion" to the script (ie fpsnum=25, fpsden=1 for 25fps video) then you probably need it.
hello_hello is offline   Reply With Quote
Old 10th May 2015, 03:45   #9  |  Link
luoyanghero
Guest
 
Posts: n/a
qtgmc only support yuv420p
  Reply With Quote
Old 10th May 2015, 07:34   #10  |  Link
8-BaLL
Registered User
 
Join Date: Jun 2012
Posts: 75
Quote:
Originally Posted by hello_hello View Post
I use threads=1 because on a couple of odd occasions I had problems without it (I think the source video was Divx one time). Anyway, it just sets the number of threads to be used for decoding and I'm not sure more threads would necessarily speed things up, given the CPU will also be pretty busy with filtering and encoding etc. I can't say I've compared speeds though. Depending on the decoder being used it mightn't have any effect anyway (if it doesn't support threaded decoding). Threads=0 means the same number of threads as you have CPU cores. The default is threads=-1 (minus one) which is the same as threads=0. Is that what you meant when you said it automatically selects threads=1?

I never rule out using DirectShowSource if other methods fail with "problem" files. Sometimes (at least with ffdshow decoding) I've found it'll keep decoding when other methods fail, but I tend to only use it for simple "start to finish" encoding. For anything else I'd use DirectShowSource to decode to a lossless file and then re-encode from there.

Oh..... and there' an issue with AVIs and ffms2 if you use it's frame rate conversion in a script. If you don't it's fine, but for some reason it tends to drop or duplicate frames now and then when it shouldn't and motion can be "jittery" in places (if a video is 25fps and you use fpsnum=25 and fpsden=1 in a script, in theory it should have no effect). I don't think it's been fixed but it only effects AVIs and I tend to go with "remuxing as MKV" for most file types before encoding anyway. I think using the frame rate conversion in each script is a good idea, maybe even more so for TV captures, because without it if there's any missing/null frames the video gets treated as variable frame rate and it can mess with the audio sync. If you add the frame rate conversion stuff to the script (ffms2 and L-Smash both have the same options) then the output should be the correct constant frame rate and frames will be duplicated as required to account for any "missing" frames. Just thought I'd mention it..... but if you have any audio sync problems or the output frame rate is slightly off, try using the frame rate conversion options. If the total frame count changes when you add the appropriate frame rate "conversion" to the script (ie fpsnum=25, fpsden=1 for 25fps video) then you probably need it.

OK thanks, I will test some different things, but so far the ffms2 doesnt run well, unfortunately. I did a test encode and ffms2 has glitched frames in the encode, every 10 seconds it repeats with a couple of glitched frames. The sourtce was remuxed in MKV. Whats weird is that the glitch happens every 10 seconds exactly.

Here is the encode example 80MB:

https://www.dropbox.com/s/u5755v225f...ffds2.mkv?dl=0

I am now trying the l-smash LWLibavVideoSource() and will report back if it worked properly.

This is the source:

https://www.dropbox.com/s/frkfx5p3x9...p-test.ts?dl=0



edit: looks like l-smash worked properly without any glitches etc.

Last edited by 8-BaLL; 10th May 2015 at 09:00.
8-BaLL is offline   Reply With Quote
Old 10th May 2015, 11:39   #11  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
Quote:
Originally Posted by luoyanghero View Post
qtgmc only support yuv420p
From the QTGMC avs file:

Input colorspaces: YV12, YUY2

Core plugins:
SSE2Tools for YUY2 support (from the earlier 0.9 version of RemoveGrain, use only SSE2Tools.dll from this version. Don't use the SSE3 version)

That's from QTGMC 3.33 (there's a link to it buried somewhere in the QTGMC thread).
The QTGMC 3.32 avs file doesn't mention SSE2Tools.dll specifically, just RemoveGrain and Repair as required plugins.

In this case though, maybe it's a good thing QTGMC wasn't working as it made it obvious the video was being converted as it was being decoded.
hello_hello is offline   Reply With Quote
Old 10th May 2015, 13:22   #12  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
Quote:
Originally Posted by 8-BaLL View Post
OK thanks, I will test some different things, but so far the ffms2 doesnt run well, unfortunately. I did a test encode and ffms2 has glitched frames in the encode, every 10 seconds it repeats with a couple of glitched frames. The sourtce was remuxed in MKV. Whats weird is that the glitch happens every 10 seconds exactly.
Interlaced h264 can be a bit difficult sometimes.

I downloaded your sample and experienced the same glitch when indexing the TS file, but after remuxing as an MKV with MKVMergeGUI and indexing the MKV instead it seemed fine.

Or if L-Smash is doing a better job.......
I don't work with interlaced h264 very much myself. Hardly at all, actually.
hello_hello is offline   Reply With Quote
Old 10th May 2015, 21:38   #13  |  Link
8-BaLL
Registered User
 
Join Date: Jun 2012
Posts: 75
@hello_hello-

well Deinterlacing is THE main PITA to be honest with you. Encoding progressive stuff is like 20x less work

For example right now Im encoding a tv show (tv series) Ive captured from hdtv as well, but the source is not interlaced.

Its 1080i source as well and Im just resizing it to 720p and my system encodes at 11.6 fps with crf18 and "very slow" x264 preset.

The test file above, using "QTGMC preset=very slow" and exactly the same x264 preset, my system encodes with 1.72 fps. Thats 5-6x slower, just because of interlacing

If it was about me, I would only allow progressive video and only square pixels and only 1 codec, beeing h264 high profile above 4.1... Would solve like 90% of all problems.
8-BaLL is offline   Reply With Quote
Old 16th May 2015, 02:16   #14  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
The trouble with QTGMC is eventually you may discover as a de-interlacer it also makes a pretty decent denoiser. Then you find you're running it in progressive mode a lot and using it almost every time you encode. There's days, if I encode without filtering I almost feel like it's a luxury..... and watching the encode progress along at 15fps or more seems a little un-real. The one advantage to running QTGMC in progressive mode is you can resize first, then add QTGMC to the script, unlike de-interlacing where you need to de-interlace first, then resize. For desnoising in progressive mode, I find the result rends to be a little more natural looking if you run QTGMC a little faster, although I tend to use QTGMC in combination with LSFMod more and more these days, which slows things down.

QTGMC(InputType=1, Preset="Medium", EzDenoise=1)
LSFMod()

No filtering
Before (resized to 720p, no filtering):


QTGMC + LSFMod


Anyway, check your CPU usage when running QTGMC. Changes are it won't be running very hard as QTGMC is a bottleneck. You can either go the multi-threaded AVIsynth route to speed it up, which is less stable, or do what I do and run more than one encode at a time. You'll still only encode at 1.72fps (or maybe a tad slower), but at least you'll be doing it twice. When encoding just one video I make a copy of the script, add Trim() to the each so each script is encoding half the video, then I run them together, When it's done, the encoded video can easily be appended with MKVMergeGUI (make sure to add --stitchable to x264 command line so you won't have any problems appending them).
hello_hello is offline   Reply With Quote
Old 16th May 2015, 18:23   #15  |  Link
ChiDragon
Registered User
 
ChiDragon's Avatar
 
Join Date: Sep 2005
Location: Vancouver
Posts: 600
Your second image nuked the detail on the rope that's hanging near 45 degrees.

Film grain is not your enemy.
ChiDragon is offline   Reply With Quote
Old 17th May 2015, 02:01   #16  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
Quote:
Originally Posted by ChiDragon View Post
Your second image nuked the detail on the rope that's hanging near 45 degrees.
I think that's a QTGMC side effect. I'd have to test with a different denoiser to be sure, but that rope "detail" kind of resembles combing from interlaced video, and given the angle of the rope and QTGMC being a de-interlacer...... so I'm thinking that's probably not a typical side effect. It didn't remove the detail from the rope anywhere else.

But yeah, any denoiser is going to have some negative effect. Blurring where there's motion seems to be the worst problem and QTGMC's noise removal to blurring ratio is pretty good. Plus it helps stabilise the picture and reduce compression artefacts such as blocking. In this case it removed some detail from 2% of the video while improving the other 98%, and it's nothing I imagine I would have noticed while watching it, as opposed to the noise which I can't help but notice, so I'm not complaining.

Quote:
Originally Posted by ChiDragon View Post
Film grain is not your enemy.
Yes it is. Unless there's so little of it I can't see it without putting my nose on the screen. Do you actually like looking at noise? Re-encoded noise tends to become more distracting as it looks less natural (noise turned into "dancing blocks" drives me nutty), it requires a higher bitrate for a given quality, and if I do want to apply a little bit of sharpening, ideally I'd prefer not to sharpen noise. Imagine if film had been invented as a completely noise free medium, and someone came along and said "I've got an idea, lets cover the picture with all this grainy stuff"......
Roll on digital.... where I suppose initially noise will be added artificially for backwards compatibility.

It helps to reduce banding, but what other redeeming features does noise have?

I do get your point, but myself, I don't like noise.

Last edited by hello_hello; 17th May 2015 at 09:24.
hello_hello is offline   Reply With Quote
Old 17th May 2015, 12:30   #17  |  Link
creaothceann
Registered User
 
Join Date: Jul 2010
Location: Germany
Posts: 359
1. Noise is an inherent property of the film medium. Objectively speaking that's the most authentic representation of the original, and everything else is subjective.
2. Every processing of the material reduces quality, either by the filter or by the recompression (unless you want to go lossless).

Sure you can remove the grain (and some details), SVP it to 60fps etc. but it's an old film and should look like an old film from the theater. Degraining removes that feeling.

EDIT: Just look at these nice shots: 1 2 3 4 5 6 7 8 9 10

Seeing noise like that means that you're looking at a high-quality video.

Last edited by creaothceann; 19th May 2015 at 16:56.
creaothceann is offline   Reply With Quote
Old 19th May 2015, 09:57   #18  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
I tried, but I'm getting a "don't hotlink" message for each one.
hello_hello is offline   Reply With Quote
Old 19th May 2015, 16:56   #19  |  Link
creaothceann
Registered User
 
Join Date: Jul 2010
Location: Germany
Posts: 359
Fixed.
creaothceann is offline   Reply With Quote
Old 19th May 2015, 22:00   #20  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,610
I don't SVP but I can see the argument for using it. TVs cant display film the way it's displayed in a theatre so you get that "sample and hold" effect. That's the one that makes motion look "jittery" isn't it? Similar to film's "strobing" when a camera moves at just the right speed. So the "looking like an old film displaying as it would in the theatre" argument breaks down a little there.

If it makes you feel any better I only remove as much noise as I can without having a noticeable blurring effect. For very fine noise often QTGMC without it's noise removal enabled does the job as it "stabilises", but for heaver noise "stabilising" looks unnatural and two much noise removal blurs, so my compromise is to only use a little of QTGMC's noise filtering and a faster preset which doesn't stabilise as much and the end result is mostly as close as you'll probably get to reducing the noise without things starting to look unnatural. Even in my previous screenshot you can see not all the noise was removed, but it's reduced quite a bit. Much of it's probably personal preference.
hello_hello is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:34.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.