Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th June 2017, 08:12   #1321  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,181
Found something, there are two things.
First, I don't know if it is allowed or not: TDecimate class constructor is calling a GetFrame for the child clip to establish whether there are special encoded "hints" in the previous frame.
When I moved this decision into TDecimate's GetFrame, the x264 problem disappears.

The problem is also disappearing when I'm using my mvtools2 build 2.7.0.22d (this was built with ICL), but appears when 2.7.1.22 (which had a bunch of changes compared to 2.7.0.22d besides being built by VS2015) or newer mvtools2 is present.
I suspect that some resources is not being freed up or handled properly when the filter (e.g MAnalyze) is called before the whole filter graph is evaluated by avisynth.

Anyway, I think that moving frame hint detection out from class constructor is solving this problem.

Question: decimation does not seem to like multithreading, so it should be a MT_SERIALIZED filter in AVS+ terminology? (in 1.08 there is no autoregistration for this filter yet)
pinterf is offline   Reply With Quote
Old 8th June 2017, 09:54   #1322  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,404
IIRC, it was always recommended to use SetMTMode(5) with TDecimate to disable multithreading that function. I'm not even sure if I got it working without that line when I used AVS 2.6 MT.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 8th June 2017, 10:14   #1323  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,181
Quote:
Originally Posted by hello_hello View Post
If TIVTC is updated again sometime in the future, the version numbers it writes to the output files it creates (2 pass or VFR mode) could do with an update too.

The TFM metrics file starts like this:
#TFM v1.0.4 by tritical
[...]
The TDecimate output file looks like this:
#TDecimate v1.0.3 by tritical
tritical versioned the modules individually, since the logic did not change, I left them as-is.
pinterf is offline   Reply With Quote
Old 8th June 2017, 14:45   #1324  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,181
New build

Download TIVTC 1.0.9

Code:
**v1.0.9 (20170608)**
- Fix (workaround): Move frame hints detection from constructor into the first GetFrame (x64 build with 64 bit x264 crash under mysterious circumstances)
- Filters autoregister themselves as MT_SERIALIZED for Avisynth+, except MergeHints (MT_MULTI_INSTANCE)
  Note: for proper serialized behaviour under Avisynth+ MT, please use avs+ r2504 or later.
Thanks downloaded for the report.
pinterf is offline   Reply With Quote
Old 13th July 2017, 01:22   #1325  |  Link
XStylus
Registered User
 
XStylus's Avatar
 
Join Date: Jan 2004
Posts: 22
Thanks so much for the updates. By any chance is there potential for support for other color spaces (such as YV16, YV24) at some point in the future?
XStylus is offline   Reply With Quote
Old 13th July 2017, 12:14   #1326  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,612
pinterf,

I've been using TDecimate, mode=2 for 2 pass decimation, and now and then I find the second pass won't run and I get an access violation error. I've only checked once, but it also appears to be a problem with the earlier versions of TIVTC, so it's not something you've caused. Oddly though, every time it's happened I've been able to work around it by reducing the frame count by one with Trim() and deleting the entry for the last frame from the metrics file, or by adding a frame to the total and creating another one at the end of the metrics file. Changing mode 2 to mode 7 also fixes it, but it's not a two pass method.

If I can create a sensible sample (something of a decent size for uploading) are you interested in having a look? It might take me a while to create a small sample through trial and error, which is why I'm asking first.

Cheers.
hello_hello is offline   Reply With Quote
Old 17th July 2017, 09:34   #1327  |  Link
B.F.
Registered User
 
B.F.'s Avatar
 
Join Date: Jul 2004
Location: Russia, Novosibirsk
Posts: 92
Hm, as far as I remember original TIVTC can't detect purely horizontal motion.
Is it fixed in the new builds or not?
B.F. is offline   Reply With Quote
Old 17th July 2017, 16:23   #1328  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,181
Quote:
Originally Posted by hello_hello View Post
pinterf,
If I can create a sensible sample (something of a decent size for uploading) are you interested in having a look? It might take me a while to create a small sample through trial and error, which is why I'm asking first.

Cheers.
Thanks, then please prepare that sample, I can check it.
(When I disappear it's because of the summer holidays, then extra work because of holiday and I better run and prepare for marathon than sitting behind the computer in this nice weather)
pinterf is offline   Reply With Quote
Old 18th July 2017, 15:26   #1329  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,612
Run a marathon???? I'm unusually out of breath by the time I get to the end of the driveway to collect the mail.

Everything below is a combination of myself, Windows XP, Avisynth 2.6 and TIVTC 1.09, although I think it also applies to the original TIVTC (1.05, or whatever version that is). I've only tested the old version once but the result was the same.

I think I've managed to dwindle a script down to one that'll produce the error regardless of the source. In case it matters, I was using a 4:3 NTSC DVD as the source, but I've tested it with three different NTSC DVDs with the same result each time, so hopefully it's source independent.
Here's the steps required for error reproduction. Sorry if some of it's a bit obvious, but just to be clear....

This is the script:

Quote:
DGDecode_mpeg2source("D:\test.d2v")
Yadif(mode=1)
AssumeFPS(60000,1001)
Trim(0, 39582)
TDecimate(mode=2, rate=50, Input="D:\metrics.txt")
AssumeFPS(50)
And here's the metrics file (same file, different links):
metrics.zip
http://www.filedropper.com/metrics

When you first run the script, you'll bump into a crc32 mismatch error such as the following:



You can try using batch=true for TDecimate to bypass the crc32 check but when I started testing I thought it stopped the problem from occurring a couple of times, although maybe I did something silly. If it does, you might need to manually change the crc32 at the top of the metrics file to match your source.

Re-opening the script should then take you directly to the problem I've been encountering (now and then).



TDecimate doesn't seem to care if there's less frames in the metrics file than in the video itself, although it gets upset if there's more, but anyway.... to work around the problem you can simply delete the reference to the last frame in the metrics file. You can also reduce the trim range by a frame to match and the problem shouldn't return. ie Trim(0, 39581)

It seems the trick is to have just the right number of frames. If that's the cause there's no doubt more than one magic number, but if you keep the metrics file as it was originally, and move the trim range up by 6 frames ie Trim(6, 39588), you should still bump into the same access violation error.

Thanks.

PS. Did you happen to notice this? I'm just wondering if it's a bug or me having a silly. Thanks.
https://forum.doom9.org/showthread.p...64#post1810264

Last edited by hello_hello; 18th July 2017 at 16:11.
hello_hello is offline   Reply With Quote
Old 18th July 2017, 19:35   #1330  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Radical, think-outside-the-box, "out there" suggestion: have you tried using a metrics file that actually matches the clip you're operating on?
TheFluff is offline   Reply With Quote
Old 19th July 2017, 11:29   #1331  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,612
Believe it or not, that's how the metrics file was created originally, so it did match the source, but it doesn't seem to matter as it results in an access violation error whether I use the same source or a different one, so I went with the "no need to upload video" option as my internet connection is slow.

Aside from having to change the crc32 or use batch=true, how would TDecimate know the source is different? If it checked the whole video, wouldn't that defeat the purpose of running the first pass and creating the metrics file in the first place?

Last edited by hello_hello; 19th July 2017 at 20:06.
hello_hello is offline   Reply With Quote
Old 15th August 2017, 00:18   #1332  |  Link
zambelli
Doom9ing since 2001
 
zambelli's Avatar
 
Join Date: Oct 2001
Location: Seattle, WA, USA
Posts: 1,996
I'm trying to use TDecimate to remove a progressive 3:2 pulldown (where ABCD frames are duplicated as AAABBCCCDD frames, for example, such as in 720p60 broadcasts of 24p content).

I'm using this:
Code:
TDecimate(mode=1, cycleR=6, cycle=10)
It seems to work well, but based on the original burned-in timecode I can see that it's occasionally dropping entire frames from the original 24p sequence.

Is there a better combination of parameters I should be using for progressive 3:2 pulldown removal? Is there a better plugin than TIVTC for that specific use case?
zambelli is offline   Reply With Quote
Old 15th August 2017, 01:07   #1333  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,229
For every 5 frames you need to return 2, say the second and fifth frames. This can be done with selectevery function but I'm not sure of the syntax.
burfadel is offline   Reply With Quote
Old 15th August 2017, 01:48   #1334  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,969
How about SelectEven().TDecimate() ?

This works on 24p in 59.94p broadcasts here, but not sure why you would drop original frames in that first case - perhaps you have irregular cadence issues ? or source filter issues ?
poisondeathray is offline   Reply With Quote
Old 15th August 2017, 01:54   #1335  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 738
Here are several ways to perform IVTC (included TIVTC): Here.
__________________
By law and justice!

GMJCZP's Arsenal
GMJCZP is offline   Reply With Quote
Old 15th August 2017, 10:51   #1336  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 6,462
Quote:
Originally Posted by burfadel View Post
For every 5 frames you need to return 2, say the second and fifth frames. This can be done with selectevery function but I'm not sure of the syntax.
SelectEvery(5, 1, 4)

But the pattern must be the same along all the movie.
__________________
BeHappy, AviSynth audio transcoder.
tebasuna51 is offline   Reply With Quote
Old 15th August 2017, 12:47   #1337  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,612
Quote:
Originally Posted by zambelli View Post
I'm trying to use TDecimate to remove a progressive 3:2 pulldown (where ABCD frames are duplicated as AAABBCCCDD frames, for example, such as in 720p60 broadcasts of 24p content).

I'm using this:
Code:
TDecimate(mode=1, cycleR=6, cycle=10)
It seems to work well, but based on the original burned-in timecode I can see that it's occasionally dropping entire frames from the original 24p sequence.

Is there a better combination of parameters I should be using for progressive 3:2 pulldown removal? Is there a better plugin than TIVTC for that specific use case?
You can try stretching out the cycle.

TDecimate(mode=1, cycleR=12, cycle=20)

Or not specifying one:

TDecimate(mode=7, rate=23.976) # or rate=24, depending on the source frame rate.

Mode 7 usually does a good job, but if you find it's not getting it quite right, two passes generally does it better.

1st pass something like:

TDecimate(mode=4, rate=23.976, Output="D:\metrics.txt")

2nd pass:

TDecimate(mode=2, rate=23.976, Input="D:\metrics.txt")

A few posts up I reported a problem I was having with two pass mode. I've experienced it a couple more times since then. If you start the second pass and get an access violation error, open the metrics file and delete the very last entry. For whatever reason, that always fixes it.

I haven't used it this way, but the help file says mode 2 can produce the same result with one pass as it would with two passes if you set m2PA=true. It reads ahead quite a bit though, so a script will initially take a while to open, and encoding could appear to stall at times when one "cycle" finishes and TDecimate reads ahead again. When there's no metrics file specified, mode 2 normally reads ahead by a maximum of 100 frames. m2PA=true over-rides that.

Last edited by hello_hello; 20th August 2017 at 12:22. Reason: Corrected the info in the last paragraph
hello_hello is offline   Reply With Quote
Old 23rd December 2017, 01:30   #1338  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,612
I have a question, and while I imagine the answer will be "too much work" or something similar, you never know if you don't ask.

Currently TIVTC can output combinations of 23.976fps and 29.970fps in variable frame rate mode, but would it be possible to have it output combinations of 23.976 and 59.94 instead (while bob de-interlacing the interlaced sections)? TDecimate would probably need an update to give it a PAL hybrid mode, or so it's current hybrid mode would know not to decimate PAL, but I assume even for PAL sources, TDecimate would still be needed to create a timecodes file.
hello_hello is offline   Reply With Quote
Old 28th December 2017, 16:10   #1339  |  Link
8day
Registered User
 
Join Date: Jun 2009
Posts: 10
Anyone had this issue with TDecimate that it ignores sections of <5 frames defined as video (possibly film as well)? I tried to use TDecimate to create VFR video, but I found that it ignores, or even shrinks some segments (possibly due to scenecuts, but that's less important). E.g., when I supplied these overrides to test clip with TDecimate(mode=5, hybrid=2):
Code:
10,11 v
20,22 v
30,33 v
40,44 v
50,55 v
60,99 v
it generated these timestamps:
Code:
0,31,23.976000
37,40,23.976000
46,49,23.976000
90,826,23.976000
Obviously there should've been more segments. So, is this normal? Because after re-reading TDecimate helpme I noticed that tritical put a strong accent on ranges and cycles, i.e. most likely it works on per-cycle basis...

hello_hello, I face(d) a similar issue with NTSC. The biggest problem with doing bob-deinterlacing is the one I posted. If it can be solved, most likely there's no need for a huge rewrite (~20 simple lines, but then we'll need extra plugin akin to Remap, and extra script to generate EDL and new timestamps), otherwise, and I suspect it's the sad truth, pretty big chunk of logic must be rewritten/introduced.

BTW, VapourSynth's VIVTC has much smaller code base (~500 lines vs ~5500), so despite how very bad this may be, it's possible to write simpler decimator based on VDecimate.

Last edited by 8day; 28th December 2017 at 20:03. Reason: Added line about VIVTC.
8day is offline   Reply With Quote
Old 28th December 2017, 22:14   #1340  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,612
Quote:
Originally Posted by 8day View Post
Anyone had this issue with TDecimate that it ignores sections of <5 frames defined as video (possibly film as well)? I tried to use TDecimate to create VFR video, but I found that it ignores, or even shrinks some segments (possibly due to scenecuts, but that's less important). E.g., when I supplied these overrides to test clip with TDecimate(mode=5, hybrid=2)
I think it's definitely about cycles, although I'm not sure I understand the logic behind specifying a couple of consecutive frames (or even 4 consecutive frames) as being video, because for TDecimate, isn't the difference between video and film frames effectively the presence or absence of a duplicate frame in each group of five? So the question would be.... did TDecimate remove any of the frames you specified as video?

For example, you specified frames 10 and 11 as video but that covers 3 cycles of 5 frames, so if frames 4, 9, and 13 were duplicates and removed, frames 10 and 11 would now have a duration of roughly 41.7ms instead of their original "video" durations of 33.4ms, as they've become part of a film cycle, but as long as they're still there....

Plus if specifying frames 10 and 11 as video forced frames 6 thru 15 to be considered video (assuming cycles of five) and there were a couple of duplicates amongst them, would keeping the duplicates be better than removing them and calling the remaining frames film? If TDecimate did keep the original "video durations" for 10 and 11 while removing frame 12 because it was a duplicate, that'd mean frames 13 and 14 would have to be (taking a guess) 20fps to complete the cycle.

I'm not answering your question as such.... just thinking about it out loud....

Quote:
Originally Posted by 8day View Post
hello_hello, I face(d) a similar issue with NTSC. The biggest problem with doing bob-deinterlacing is the one I posted.
I've worked out a system for doing it manually, including creating a timecodes file, and it works well, but it can be quite time consuming when there's a large number of film and video sections, although I've never wanted to bob de-interlace just a few consecutive frames. It's always "sections" of film and "sections" of video, and for the film sections you really do need to stick to multiples of five, given after decimation, four film frames = five video frames in duration, otherwise you'd have to start fiddling with frame rates for individual sections of film or video, or add/remove frames in video sections to compensate for any change in duration after decimating the film sections etc.... and that'd make it even harder.

Last edited by hello_hello; 28th December 2017 at 23:17.
hello_hello is offline   Reply With Quote
Reply

Tags
tdeint, tivtc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:46.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.