Log in

View Full Version : TDeint and TIVTC


Pages : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 [27] 28 29 30 31 32 33 34

pinterf
27th April 2017, 13:13
With the great help of real.finder:

New release:
TIVTC 1.0.7 (x86/x64) (https://github.com/pinterf/TIVTC/releases/tag/v1.0.7)
Please, still treat it as beta (due to moving all inline asm parts to simd intrinsics, plus new SSE2 parts), but of course feedback is appreciated.

**v1.0.7 (20170427)**
- fix crash in FieldDiff (in new SIMD SSE2 rewrite)

**v1.0.6 (20170421) - pinterf**
- project migrated to VS 2015
- AVS 2.6 interface, no Avisynth 2.5.x support
- some fixes
- x64 port and readability: move all inline asm to simd intrinsics or C
- supports and requires SSE2
- MMX and ISSE is not supported, but kept in the source code for reference
- source code cleanups

**v1.0.5 (2008) - tritical**
- see old readmes

pinterf
27th April 2017, 17:08
Tested four versions:
1.0.5 original
1.0.5 vc10 rebuild
1.0.7 with inline asm (my internal test build)
1.0.7 with simd intrinsics (release)
All 32 bits for comparison

The latter two is bit identical which is good news.

Speedwise: all same, vc10 build is slower by 4-5%.

The two 1.0.5 builds gave slightly different result.

1.0.7 build is showing differences to 1.0.5 on frame by frame comparison, they are almost the same, but it general 1.0.7 gave less false combed frames.

Tested on an interlaced anime clip, that I was given once in PM.

pinterf
30th April 2017, 07:34
New TIVTC build.
TIVTC 1.08 (beta (https://github.com/pinterf/TIVTC/releases))

(simd intrinsics rewrite fix)
Thanks to mizumanju for the report.

**v1.0.8 (20170429)**
- Fix: TFM PP=2 and PP=5 (Blend deint)

downloaded
5th June 2017, 15:18
When using with QTGMC with presets Medium to Very Fast it just crashes for me.

QTGMC with presets Slow+ works fine.

Regardless QTGMC preset works fine using http://www.mediafire.com/file/i2qtli1mxik/TIVTC_3-13-2010.rar .

cork_OS
5th June 2017, 16:20
When using with QTGMC with presets Medium to Very Fast it just crashes for me.
No problems with Xeon E3-1270 V3.

Groucho2004
5th June 2017, 17:12
When using with QTGMC with presets Medium to Very Fast it just crashes for me.Define crash. No error message?

downloaded
5th June 2017, 23:26
Preview on Avspmod trows a generic error "%d format: a number is required, not float." but it previews anyway.

Launching x264 on cli just makes process stop responding without any error msg. I'm available to debug it, but i need some help on what to do.

Groucho2004
6th June 2017, 07:05
Preview on Avspmod trows a generic error "%d format: a number is required, not float." but it previews anyway.I think that's an error Python throws so this is probably some Avspmod issue, not sure.

Launching x264 on cli just makes process stop responding without any error msg. I'm available to debug it, but i need some help on what to do.You should try running your script with AVSMeter (https://forum.doom9.org/showthread.php?t=173259) to see if that reveals any problems.

downloaded
6th June 2017, 13:10
AVSMeter 2.5.7 (x64) - Copyright (c) 2012-2017, Groucho2004
AviSynth+ 0.1 (r2504, MT, x86_64) (0.1.0.0)

Number of frames: 30508
Length (hh:mm:ss.ms): 00:21:12.438
Frame size: 1274 x 712
Framerate: 23.976 (24000/1001)
Colorspace: i420

Frames processed: 501 (1000 - 1500)
FPS (min | max | average): 0.423 | 155004 | 17.31
Memory usage (phys | virt): 1306 | 1326 MiB
Thread count: 22
CPU usage (average): 32%

Time (elapsed): 00:00:28.943

Works fine. And yet it still crashes x264.

downloaded
6th June 2017, 13:13
AVSMeter 2.5.7 (x64) - Copyright (c) 2012-2017, Groucho2004
AviSynth+ 0.1 (r2504, MT, x86_64) (0.1.0.0)

Number of frames: 30508
Length (hh:mm:ss.ms): 00:21:12.438
Frame size: 1274 x 712
Framerate: 23.976 (24000/1001)
Colorspace: i420

Frames processed: 501 (1000 - 1500)
FPS (min | max | average): 0.423 | 155004 | 17.31
Memory usage (phys | virt): 1306 | 1326 MiB
Thread count: 22
CPU usage (average): 32%

Time (elapsed): 00:00:28.943

Works fine. And yet it still crashes x264.

pinterf
6th June 2017, 13:49
Yes,
AVSMeter -avsinfo
would definitely help. And that would answer some other questions, too, such as Avs version, 32 or 64 bits, etc.
My 1.08 build is SSE2 only, requires it and does not use higher processor features than that.

downloaded
6th June 2017, 15:25
AVSMeter 2.5.7 (x64) - Copyright (c) 2012-2017, Groucho2004

VersionString: AviSynth+ 0.1 (r2504, MT, x86_64)
VersionNumber: 2.60
File / Product version: 0.1.0.0 / 0.1.0.0
Interface Version: 6
Multi-threading support: Yes
Avisynth.dll location: C:\xxxxxxxxxxxxxxxx\AviSynth.dll
Avisynth.dll time stamp: 2017-06-03, 21:06:38 (UTC)


[Plugin errors/warnings]
___________________________________________________________________________________________________________________________________________________________

No plugin directory references found in the registry.
Plugin auto-loading disabled.
___________________________________________________________________________________________________________________________________________________________

AVSMeter 2.5.7 (x64) - Copyright (c) 2012-2017, Groucho2004
AviSynth+ 0.1 (r2504, MT, x86_64) (0.1.0.0)

Number of frames: 30508
Length (hh:mm:ss.ms): 00:21:12.438
Frame size: 1274 x 712
Framerate: 23.976 (24000/1001)
Colorspace: i420

Frames processed: 501 (1000 - 1500)
FPS (min | max | average): 0.423 | 155004 | 17.31
Memory usage (phys | virt): 1306 | 1326 MiB
Thread count: 22
CPU usage (average): 32%

Time (elapsed): 00:00:28.943

Works fine, but it still crashes with x264.

Groucho2004
6th June 2017, 15:57
@downloaded
Hard to say why it crashes with x264 but it doesn't seem to be a problem in the filter chain since it works with AVSMeter.

As for the missing plugin directory references - How exactly did you install Avisynth+ and how did you update to the latest version?

You should do a clean install as I suggested here (https://forum.doom9.org/showthread.php?p=1808637#post1808637).

pinterf
6th June 2017, 16:06
I suppose you are using 64 bit x264. Still, since you have no autoloaded plugins, I cannot tell from this avsinfo the DLL versions you are exactly using.
I tried to encode a qtgmc'd clip with similar frame dimensions and frame rate, but no crash at all. x264 x64 version of r2665

downloaded
6th June 2017, 16:54
@downloaded
Hard to say why it crashes with x264 but it doesn't seem to be a problem in the filter chain since it works with AVSMeter.

As for the missing plugin directory references - How exactly did you install Avisynth+ and how did you update to the latest version?

You should do a clean install as I suggested here (https://forum.doom9.org/showthread.php?p=1808637#post1808637).

I didnt install, i never did. I just use the dll directly on same dir as x264 binary and inside avspmod dir.

I suppose you are using 64 bit x264. Still, since you have no autoloaded plugins, I cannot tell from this avsinfo the DLL versions you are exactly using.
I tried to encode a qtgmc'd clip with similar frame dimensions and frame rate, but no crash at all. x264 x64 version of r2665

Yes, i'm using 64bit x264. I use AddAutoloadDir inside avs script instead of load plugins. I forgot a detail, maybe its important. Height its not mod4 so i addborders first and crop after decimate. Script follows:

SetFilterMTMode("DEFAULT_MT_MODE", 2)
AddAutoloadDir("C:/xxxxxxxxxxxxxxxxxxxx")
ffvideosource("C:\xxxxxxxxxxxxxxxxxxx.mkv")
ConvertToYV12()
AddBorders(0, 2, 0, 0)
QTGMC(Preset="Very Fast",EdiThreads=2).selecteven()
tdecimate(cycleR=1,cycle=5)
Crop(2,2,-4,-2)
Prefetch(4)

Groucho2004
6th June 2017, 19:46
I use AddAutoloadDir inside avs script instead of load plugins.Ah, OK.

SetFilterMTMode("DEFAULT_MT_MODE", 2)
AddAutoloadDir("C:/xxxxxxxxxxxxxxxxxxxx")
ffvideosource("C:\xxxxxxxxxxxxxxxxxxx.mkv")
ConvertToYV12()
AddBorders(0, 2, 0, 0)
QTGMC(Preset="Very Fast",EdiThreads=2).selecteven()
tdecimate(cycleR=1,cycle=5)
Crop(2,2,-4,-2)
Prefetch(4)
Have you tried without the Prefetch statement?

downloaded
6th June 2017, 23:55
Yes, same exact problem.

pinterf
7th June 2017, 07:31
Ok, now I know your script, I will run tests with those parameters

EDIT: successfully crashed after changing x264 version to r2762.
EDIT 2: Exception thrown at 0x000000014001D63F in x264_64.exe: 0xC0000005: Access violation reading location 0x00000000C101AD62.
EDIT 3: latest Komisar build x264.2833.x86_64.exe is working again, please try that one.

downloaded
7th June 2017, 09:30
This is the one i was using and crashing:

x264 0.150.2833 df79067
(libswscale 4.7.101)
(libavformat 57.72.101)
(ffmpegsource 2.23.0.0)
built by Komisar on May 26 2017, gcc: 4.9.2 (multilib.generic.Komisar)
x264 configuration: --bit-depth=8 --chroma-format=all
libx264 configuration: --bit-depth=8 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later

Tested with this one and same issue:

x264 0.150.2833kMod df79067
(libswscale 4.7.101)
(libavformat 57.72.101)
(ffmpegsource 2.23.0.0)
built by Komisar on May 26 2017, gcc: 4.9.2 (multilib.generic.Komisar)
x264 configuration: --bit-depth=8 --chroma-format=all
libx264 configuration: --bit-depth=8 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later

hello_hello
7th June 2017, 18:44
If TIVTC is updated again sometime in the future, the version numbers it writes to the output files it creates (2 pass or VFR mode) could do with an update too.

The TFM metrics file starts like this:

#TFM v1.0.4 by tritical
field = top
crc32 = 385d356e
0 h + [256]

The TDecimate output file looks like this:

#TDecimate v1.0.3 by tritical
crc32 = f76ebc82, blockx = 32, blocky = 32, chroma = T
21885 12119 361525

And the timecodes file created by TDecimate:

# timecode format v1
Assume 29.970030
# TDecimate v1.0.3 by tritical
# Mode 5 - Auto-generated mkv timecodes file

pinterf
8th June 2017, 08:12
Found something, there are two things.
First, I don't know if it is allowed or not: TDecimate class constructor is calling a GetFrame for the child clip to establish whether there are special encoded "hints" in the previous frame.
When I moved this decision into TDecimate's GetFrame, the x264 problem disappears.

The problem is also disappearing when I'm using my mvtools2 build 2.7.0.22d (this was built with ICL), but appears when 2.7.1.22 (which had a bunch of changes compared to 2.7.0.22d besides being built by VS2015) or newer mvtools2 is present.
I suspect that some resources is not being freed up or handled properly when the filter (e.g MAnalyze) is called before the whole filter graph is evaluated by avisynth.

Anyway, I think that moving frame hint detection out from class constructor is solving this problem.

Question: decimation does not seem to like multithreading, so it should be a MT_SERIALIZED filter in AVS+ terminology? (in 1.08 there is no autoregistration for this filter yet)

Boulder
8th June 2017, 09:54
IIRC, it was always recommended to use SetMTMode(5) with TDecimate to disable multithreading that function. I'm not even sure if I got it working without that line when I used AVS 2.6 MT.

pinterf
8th June 2017, 10:14
If TIVTC is updated again sometime in the future, the version numbers it writes to the output files it creates (2 pass or VFR mode) could do with an update too.

The TFM metrics file starts like this:
#TFM v1.0.4 by tritical
[...]
The TDecimate output file looks like this:
#TDecimate v1.0.3 by tritical

tritical versioned the modules individually, since the logic did not change, I left them as-is.

pinterf
8th June 2017, 14:45
New build

Download TIVTC 1.0.9 (https://github.com/pinterf/TIVTC/releases/tag/v1.0.9)

**v1.0.9 (20170608)**
- Fix (workaround): Move frame hints detection from constructor into the first GetFrame (x64 build with 64 bit x264 crash under mysterious circumstances)
- Filters autoregister themselves as MT_SERIALIZED for Avisynth+, except MergeHints (MT_MULTI_INSTANCE)
Note: for proper serialized behaviour under Avisynth+ MT, please use avs+ r2504 or later.


Thanks downloaded for the report.

XStylus
13th July 2017, 01:22
Thanks so much for the updates. By any chance is there potential for support for other color spaces (such as YV16, YV24) at some point in the future?

hello_hello
13th July 2017, 12:14
pinterf,

I've been using TDecimate, mode=2 for 2 pass decimation, and now and then I find the second pass won't run and I get an access violation error. I've only checked once, but it also appears to be a problem with the earlier versions of TIVTC, so it's not something you've caused. Oddly though, every time it's happened I've been able to work around it by reducing the frame count by one with Trim() and deleting the entry for the last frame from the metrics file, or by adding a frame to the total and creating another one at the end of the metrics file. Changing mode 2 to mode 7 also fixes it, but it's not a two pass method.

If I can create a sensible sample (something of a decent size for uploading) are you interested in having a look? It might take me a while to create a small sample through trial and error, which is why I'm asking first.

Cheers.

B.F.
17th July 2017, 09:34
Hm, as far as I remember original TIVTC can't detect purely horizontal motion.
Is it fixed in the new builds or not?

pinterf
17th July 2017, 16:23
pinterf,
If I can create a sensible sample (something of a decent size for uploading) are you interested in having a look? It might take me a while to create a small sample through trial and error, which is why I'm asking first.

Cheers.
Thanks, then please prepare that sample, I can check it.
(When I disappear it's because of the summer holidays, then extra work because of holiday and I better run and prepare for marathon than sitting behind the computer in this nice weather)

hello_hello
18th July 2017, 15:26
Run a marathon???? I'm unusually out of breath by the time I get to the end of the driveway to collect the mail. ;)

Everything below is a combination of myself, Windows XP, Avisynth 2.6 and TIVTC 1.09, although I think it also applies to the original TIVTC (1.05, or whatever version that is). I've only tested the old version once but the result was the same.

I think I've managed to dwindle a script down to one that'll produce the error regardless of the source. In case it matters, I was using a 4:3 NTSC DVD as the source, but I've tested it with three different NTSC DVDs with the same result each time, so hopefully it's source independent.
Here's the steps required for error reproduction. Sorry if some of it's a bit obvious, but just to be clear....

This is the script:

DGDecode_mpeg2source("D:\test.d2v")
Yadif(mode=1)
AssumeFPS(60000,1001)
Trim(0, 39582)
TDecimate(mode=2, rate=50, Input="D:\metrics.txt")
AssumeFPS(50)

And here's the metrics file (same file, different links):
metrics.zip (https://files.videohelp.com/u/210984/metrics.zip)
http://www.filedropper.com/metrics

When you first run the script, you'll bump into a crc32 mismatch error such as the following:

https://image.ibb.co/jgGmVF/crc32.gif

You can try using batch=true for TDecimate to bypass the crc32 check but when I started testing I thought it stopped the problem from occurring a couple of times, although maybe I did something silly. If it does, you might need to manually change the crc32 at the top of the metrics file to match your source.

Re-opening the script should then take you directly to the problem I've been encountering (now and then).

https://image.ibb.co/jMTpja/access_violation.gif

TDecimate doesn't seem to care if there's less frames in the metrics file than in the video itself, although it gets upset if there's more, but anyway.... to work around the problem you can simply delete the reference to the last frame in the metrics file. You can also reduce the trim range by a frame to match and the problem shouldn't return. ie Trim(0, 39581)

It seems the trick is to have just the right number of frames. If that's the cause there's no doubt more than one magic number, but if you keep the metrics file as it was originally, and move the trim range up by 6 frames ie Trim(6, 39588), you should still bump into the same access violation error.

Thanks.

PS. Did you happen to notice this? I'm just wondering if it's a bug or me having a silly. Thanks.
https://forum.doom9.org/showthread.php?p=1810264#post1810264

TheFluff
18th July 2017, 19:35
Radical, think-outside-the-box, "out there" suggestion: have you tried using a metrics file that actually matches the clip you're operating on?

hello_hello
19th July 2017, 11:29
Believe it or not, that's how the metrics file was created originally, so it did match the source, but it doesn't seem to matter as it results in an access violation error whether I use the same source or a different one, so I went with the "no need to upload video" option as my internet connection is slow.

Aside from having to change the crc32 or use batch=true, how would TDecimate know the source is different? If it checked the whole video, wouldn't that defeat the purpose of running the first pass and creating the metrics file in the first place?

zambelli
15th August 2017, 00:18
I'm trying to use TDecimate to remove a progressive 3:2 pulldown (where ABCD frames are duplicated as AAABBCCCDD frames, for example, such as in 720p60 broadcasts of 24p content).

I'm using this:
TDecimate(mode=1, cycleR=6, cycle=10)

It seems to work well, but based on the original burned-in timecode I can see that it's occasionally dropping entire frames from the original 24p sequence.

Is there a better combination of parameters I should be using for progressive 3:2 pulldown removal? Is there a better plugin than TIVTC for that specific use case?

burfadel
15th August 2017, 01:07
For every 5 frames you need to return 2, say the second and fifth frames. This can be done with selectevery function but I'm not sure of the syntax.

poisondeathray
15th August 2017, 01:48
How about SelectEven().TDecimate() ?

This works on 24p in 59.94p broadcasts here, but not sure why you would drop original frames in that first case - perhaps you have irregular cadence issues ? or source filter issues ?

GMJCZP
15th August 2017, 01:54
Here are several ways to perform IVTC (included TIVTC): Here (https://forum.doom9.org/showthread.php?t=173654&highlight=tivtc+gmjczp).

tebasuna51
15th August 2017, 10:51
For every 5 frames you need to return 2, say the second and fifth frames. This can be done with selectevery function but I'm not sure of the syntax.

SelectEvery(5, 1, 4)

But the pattern must be the same along all the movie.

hello_hello
15th August 2017, 12:47
I'm trying to use TDecimate to remove a progressive 3:2 pulldown (where ABCD frames are duplicated as AAABBCCCDD frames, for example, such as in 720p60 broadcasts of 24p content).

I'm using this:
TDecimate(mode=1, cycleR=6, cycle=10)

It seems to work well, but based on the original burned-in timecode I can see that it's occasionally dropping entire frames from the original 24p sequence.

Is there a better combination of parameters I should be using for progressive 3:2 pulldown removal? Is there a better plugin than TIVTC for that specific use case?

You can try stretching out the cycle.

TDecimate(mode=1, cycleR=12, cycle=20)

Or not specifying one:

TDecimate(mode=7, rate=23.976) # or rate=24, depending on the source frame rate.

Mode 7 usually does a good job, but if you find it's not getting it quite right, two passes generally does it better.

1st pass something like:

TDecimate(mode=4, rate=23.976, Output="D:\metrics.txt")

2nd pass:

TDecimate(mode=2, rate=23.976, Input="D:\metrics.txt")

A few posts up I reported a problem I was having with two pass mode. I've experienced it a couple more times since then. If you start the second pass and get an access violation error, open the metrics file and delete the very last entry. For whatever reason, that always fixes it.

I haven't used it this way, but the help file says mode 2 can produce the same result with one pass as it would with two passes if you set m2PA=true. It reads ahead quite a bit though, so a script will initially take a while to open, and encoding could appear to stall at times when one "cycle" finishes and TDecimate reads ahead again. When there's no metrics file specified, mode 2 normally reads ahead by a maximum of 100 frames. m2PA=true over-rides that.

hello_hello
23rd December 2017, 01:30
I have a question, and while I imagine the answer will be "too much work" or something similar, you never know if you don't ask.

Currently TIVTC can output combinations of 23.976fps and 29.970fps in variable frame rate mode, but would it be possible to have it output combinations of 23.976 and 59.94 instead (while bob de-interlacing the interlaced sections)? TDecimate would probably need an update to give it a PAL hybrid mode, or so it's current hybrid mode would know not to decimate PAL, but I assume even for PAL sources, TDecimate would still be needed to create a timecodes file.

8day
28th December 2017, 16:10
Anyone had this issue with TDecimate that it ignores sections of <5 frames defined as video (possibly film as well)? I tried to use TDecimate to create VFR video, but I found that it ignores, or even shrinks some segments (possibly due to scenecuts, but that's less important). E.g., when I supplied these overrides to test clip with TDecimate(mode=5, hybrid=2):
10,11 v
20,22 v
30,33 v
40,44 v
50,55 v
60,99 v
it generated these timestamps:
0,31,23.976000
37,40,23.976000
46,49,23.976000
90,826,23.976000
Obviously there should've been more segments. So, is this normal? Because after re-reading TDecimate helpme I noticed that tritical put a strong accent on ranges and cycles, i.e. most likely it works on per-cycle basis...

hello_hello, I face(d) a similar issue with NTSC. The biggest problem with doing bob-deinterlacing is the one I posted. If it can be solved, most likely there's no need for a huge rewrite (~20 simple lines, but then we'll need extra plugin akin to Remap, and extra script to generate EDL and new timestamps), otherwise, and I suspect it's the sad truth, pretty big chunk of logic must be rewritten/introduced.

BTW, VapourSynth's VIVTC has much smaller code base (~500 lines vs ~5500), so despite how very bad this may be, it's possible to write simpler decimator based on VDecimate.

hello_hello
28th December 2017, 22:14
Anyone had this issue with TDecimate that it ignores sections of <5 frames defined as video (possibly film as well)? I tried to use TDecimate to create VFR video, but I found that it ignores, or even shrinks some segments (possibly due to scenecuts, but that's less important). E.g., when I supplied these overrides to test clip with TDecimate(mode=5, hybrid=2)

I think it's definitely about cycles, although I'm not sure I understand the logic behind specifying a couple of consecutive frames (or even 4 consecutive frames) as being video, because for TDecimate, isn't the difference between video and film frames effectively the presence or absence of a duplicate frame in each group of five? So the question would be.... did TDecimate remove any of the frames you specified as video?

For example, you specified frames 10 and 11 as video but that covers 3 cycles of 5 frames, so if frames 4, 9, and 13 were duplicates and removed, frames 10 and 11 would now have a duration of roughly 41.7ms instead of their original "video" durations of 33.4ms, as they've become part of a film cycle, but as long as they're still there....

Plus if specifying frames 10 and 11 as video forced frames 6 thru 15 to be considered video (assuming cycles of five) and there were a couple of duplicates amongst them, would keeping the duplicates be better than removing them and calling the remaining frames film? If TDecimate did keep the original "video durations" for 10 and 11 while removing frame 12 because it was a duplicate, that'd mean frames 13 and 14 would have to be (taking a guess) 20fps to complete the cycle.

I'm not answering your question as such.... just thinking about it out loud.... ;)

hello_hello, I face(d) a similar issue with NTSC. The biggest problem with doing bob-deinterlacing is the one I posted.

I've worked out a system for doing it manually, including creating a timecodes file, and it works well, but it can be quite time consuming when there's a large number of film and video sections, although I've never wanted to bob de-interlace just a few consecutive frames. It's always "sections" of film and "sections" of video, and for the film sections you really do need to stick to multiples of five, given after decimation, four film frames = five video frames in duration, otherwise you'd have to start fiddling with frame rates for individual sections of film or video, or add/remove frames in video sections to compensate for any change in duration after decimating the film sections etc.... and that'd make it even harder.

8day
29th December 2017, 21:47
I'm not sure I understand the logic behind specifying a couple of consecutive frames (or even 4 consecutive frames) as being video
It's not about small segments per se. It's that it behaves like it adjusts boundaries of overriden segments to fit mod5 blocks/detected telecine GOPs, and small segments seemed like the most obvious choice to test this. I think it will behave same way with segments of length 102, 103 and 104 frames.

So the question would be.... did TDecimate remove any of the frames you specified as video?
Yes, it did. For 10,11 #11 was removed/not used, for 20,22 #21 and for 30,33 it was #33. And that's the problem: it ignores overrides. What should've happened is if previous and/or next telecine GOP got split, incomplete part must've been shown at 29.97 fps, and the defined video segment itself should've been shown at 29.97 fps. If anything, such segments should've been larger, not smaller, and definitely not ignored completely.

hello_hello
30th December 2017, 10:56
Yes, it did. For 10,11 #11 was removed/not used, for 20,22 #21 and for 30,33 it was #33. And that's the problem: it ignores overrides. What should've happened is if previous and/or next telecine GOP got split, incomplete part must've been shown at 29.97 fps, and the defined video segment itself should've been shown at 29.97 fps. If anything, such segments should've been larger, not smaller, and definitely not ignored completely.

The frames you specify in the overrides file do have to correspond to what TDecimate considers to be a whole cycle, which seems to always begin at frame 0.

I created a metrics file where frame 12 was obviously a duplicate. Specifying "10,14 v" caused TDecimate to keep frame 12 and set a frame rate of 29.970 for those frames in the timecodes file. Specifying "9,13 v" however had no effect. Those frames were still considered to be film and frame 12 was dropped.

According to my metrics file, frames 12, 17 and 22 were duplicates. When I specified "9,23 v" for the overrides file, frames 12 & 17 were kept but not frame 22.

I'm not sure it's a problem though, as it seems it's easy enough to work around once you know you need to specify ranges of 5. To ensure all frames in the range 17 to 22 are treated as video instead of film, you'd need to specify the range as "15,24 v". Admittedly if that's how it works, and so far it seems to be, it would've been better to make that clear in the help file.

The same applies to not putting a space after the comma like so:
42, 59 v
The help file doesn't warn not to and there's no error if you do. I couldn't work out why my overrides file was being ignored completely until the penny dropped and I realised I'd added a space.

8day
30th December 2017, 17:51
hello_hello, this thought never crossed my mind, but you seem to be right, it kind of makes sense. Now I hope I'll be able to write necessary scripts in a day or two...

8day
3rd January 2018, 19:27
hello_hello, I think I've done it. It took longer than expected, but should work.

All the files, including links to custom plugins, can be found here: https://github.com/8day/avs-tivtcex All the stuff you need to know is described in the readme.md on that page.

hello_hello
7th January 2018, 03:54
I haven't checked it out properly yet but it looks interesting. I definitely will.

Out of curiosity, what do you use for this part of the phase 2 script?

# favourite filter chain for analysis of hybrid video.

Cheers.

8day
7th January 2018, 12:04
Anything that makes combing more visible and choices more easy will do. ATM I can't share my filter chain for various reasons (I'll share it later, when I'll finish another project, which hopefully won't take more than a month: it's mostly ready, but I want to write replacement for TDecimate), but considering what's expected from TIVTCEx, Histogram("luma") should be more than enough.

In case you didn't know, TFM generates misc stats at the end of its metrics file on combed frames, ranges, and supposedly missed combed frames. All you have to do is inspect them and create overrides file for TFM and/or TIVTCEx (I suspect that's what YATTA is built around).

Gser
18th January 2018, 21:39
Anything that makes combing more visible and choices more easy will do. ATM I can't share my filter chain for various reasons (I'll share it later, when I'll finish another project, which hopefully won't take more than a month: it's mostly ready, but I want to write replacement for TDecimate), but considering what's expected from TIVTCEx, Histogram("luma") should be more than enough.

In case you didn't know, TFM generates misc stats at the end of its metrics file on combed frames, ranges, and supposedly missed combed frames. All you have to do is inspect them and create overrides file for TFM and/or TIVTCEx (I suspect that's what YATTA is built around).

Any chance of getting x64 versions of the new filters?

8day
18th January 2018, 22:57
I doubt it. My understanding of C++/AviSynth is relatively primitive. The only reason there's this "hack" is because TIVTC compiled straight out of the box. But I'll try to compile pinterf's mod once again and maybe I'll get lucky this time... Otherwise you'll have to ask pinterf to copy this hack and recompile DLL with it.

pinterf
19th January 2018, 09:37
I doubt it. My understanding of C++/AviSynth is relatively primitive. The only reason there's this "hack" is because TIVTC compiled straight out of the box. But I'll try to compile pinterf's mod once again and maybe I'll get lucky this time... Otherwise you'll have to ask pinterf to copy this hack and recompile DLL with it.
No problem adding that hack, which source files should I check for modifications? I have filecompared TDecimate and found one place, but it would be easier to know that w/o comparing all files. I can see only root and initial commits in your git repo.
EDIT: found them and integrated. Who wants to test it?

EDIT: New build: TIVTC 1.0.10 (https://github.com/pinterf/TIVTC/releases/tag/v1.0.10)
**v1.0.10 (20180119)**
- integrate new orgOut parameter for TDecimate (by 8day) (see TDecimate - READ ME.txt)

8day
19th January 2018, 17:03
Just realized why Gser was talking about multiple DLLs... Here's EDL compiled using latest AviSynth+ headers in VS2017: https://github.com/8day/avs-edl/releases/tag/0.1. But please note that I haven't tested it since I don't have AviSynth+ on my system.