TDeint and TIVTC - Page 41

tritical · 11th November 2006, 05:06

What are the numerator and denominator values for the fps of the clip being fed into tdecimate (should be able to see the values using info())? Unfortunately, the mode 2 debug output doesn't tell much about what might be happening except that it isn't getting all the way through tdecimate's constructor. For mode 2 w/ 50 fps input and 29.970 rate, it should spit out:

[4548] TDecimate: v0.9.12.5 by tritical
[4548] mode2_num = 1 mode2_den = 3 numCycles = 19 clength = 15
[4548] mode2_cfs 0 = 15
[4548] TDecimate: inframe = 0 useframe = 1 rate = 29.969999

If you remove everything after tdecimate from the script does it still crash?

EDIT:

I was able to run this script without any problems:

mpeg2source("C:\test clips\t4.d2v",idct=7,info=3)
assumefps(25,1)
ColorMatrix(hints=true,interlaced=true)
SecureBob()
TDecimate(mode=2,input="test.txt",rate=29.970,debug=true)

and the same script without stats input.

canuckerfan · 12th November 2006, 20:21

i have some dvd telecined footage with an altering field structure (most of the movie is TFF, but sometimes it will change to BFF, I know, it's bad). Is there any possibility of handling the TTF part of the footage as TFF, and the same with respect to BFF. Thanks.

tritical · 13th November 2006, 00:58

You could use mode=4 or mode=5 in tfm, both of which can handle alternating field order since they use 3-way (p/c/n) matching. The alternative is to manually find the places where the field order changes and then either use a series of trim()'s to create a new clip out of each and field match them separately or keep it as one clip and use an overrides file in tfm to change the value for order over the necessary frame ranges.

canuckerfan · 13th November 2006, 02:02

I see. So if I had TIVTC+EEDI2 called like this:

Code:

AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

Changing the modes wouldn't simply work, would it? TDeint params would have to change, right? Wondering if there's any quick way of finding out where the field order changes in the footage without manually scanning through DGIndex.

EDIT: well, initially I was using mode 6, and it left some combing in some scenes. however, mode 4 seems to leave almost none. so far, things seem fine without adjusting the TDeint params.

ChiDragon · 13th November 2006, 04:24

tritical:
I've got a video that seems to require decimation by 184-frame cycles. Since the cycles are so large, some frames are incorrectly decimated (particularly at fades to black, where it will drop a bunch of black frames and leave duplicates in the real video). Anyway, I was trying to override these parts and TDecimate appears to be ignoring the override line past a certain point. It will only read this far:

Code:

0,1000 -++++-++++-+++++-++++-++++-++++-+++++-++++-++++-++++-+++++-++++-++++-+++

Adding anything to the end of this line doesn't actually change the decimation, whether it's + or -. The 72nd frame is where it stops following the override line.

tritical · 13th November 2006, 04:52

I probably used an 80 byte buffer for reading each line so the rest gets cut off. I'll increase the size in the next release.

Chainmax · 13th November 2006, 14:42

Quote:

Originally Posted by tritical

What are the numerator and denominator values for the fps of the clip being fed into tdecimate (should be able to see the values using info())? Unfortunately, the mode 2 debug output doesn't tell much about what might be happening except that it isn't getting all the way through tdecimate's constructor. For mode 2 w/ 50 fps input and 29.970 rate, it should spit out:

[4548] TDecimate: v0.9.12.5 by tritical
[4548] mode2_num = 1 mode2_den = 3 numCycles = 19 clength = 15
[4548] mode2_cfs 0 = 15
[4548] TDecimate: inframe = 0 useframe = 1 rate = 29.969999

If you remove everything after tdecimate from the script does it still crash?
...

I'll try it and report back ASAP.

Livesms · 13th November 2006, 21:19

What it the defference beatween

Code:

AssumeTFF()
interp = separatefields().selecteven().EEDI2(field=1)
tdeint(order=1,field=1,edeint=interp)

Code:

AssumeTFF()
Interp = SeparateFields().EEDI2(field=3)
TDeint(mode=1,order=1,edeint=Interp).selecteven()

Code:

AssumeTFF()
edeintted = SeparateFields().EEDI2(field=-2)
TDeint(mode=1,order=-1,full=false,edeint=edeintted).Selecteven()

ChiDragon · 13th November 2006, 23:24

I tried increasing the fgets value from 80 to 300 myself, and now it's actually crashing when the ovr line is longer than 103 chars.

tritical · 14th November 2006, 09:35

@canuckerfan
Sorry I didn't reply to your last post earlier, I must of missed it. You're correct on TDeint not working correctly in that script if the field order changes. I was thinking of tfm by itself (its pping doesn't depend on field order). If it is deinterlacing only a few frames it'll probably be fine though. I don't know of any automatic way to determine where the field order changes. It's interesting that you get field order changes without dgindex finding illegal tff flag transitions in the stream... what version of dgindex are you using?

@Livesms
The first two scripts are logically equivalent and should run at the same speed and produce the same output. The third script is the same as the first two except for the full=false part (it will check each input frame to see if it's combed or not).

@ChiDragon
That's strange that it crashes. I changed all the fgets() calls in tdecimate.cpp/tfm.cpp/tfmpp.cpp from 80 to 1024 and increased the linein[] arrays to 1024 bytes, and everything seemed to work fine. I did a short test with cycle=120, cycleR=2, and the ovr file marking frames 96/116 to drop and it worked fine. What are the cycle/cycleR values you are using? and what are the lines in the ovr file?

ChiDragon · 14th November 2006, 12:33

Whoops, it never occurred to me to check the linein array declaration! I was busy checking over all the ovrArray stuff... As you may have guessed, I'm not actually a programmer. :P Changing that fixed it of course, thanks.

Weird that it managed to work up until the 104th byte though, when it was only set for 80...

Livesms · 15th November 2006, 10:10

Quote:

Originally Posted by tritical

@Livesms
The first two scripts are logically equivalent and should run at the same speed and produce the same output. The third script is the same as the first two except for the full=false part (it will check each input frame to see if it's combed or not).

So what one can you reccomnd for deinterlace?

Or maybe you have some other variations?

ChiDragon · 15th November 2006, 12:50

tritical, would it be possible to add a parameter such that no 2 dups may be closer to each other than "n" (when cycleR > 1)?

I think this would solve the problem I'm having with multiple drops at black frames, and dropping the wrong frames in general with a large cycle of this kind. I've had to do a lot of overrides so far since it's often dropping stationary new frames instead of the actual dups. I know with this video that there should never be two real dups less than 5 frames apart.

I was gonna attempt this for myself so I didn't have to bug you again, but I couldn't even find the specific code that takes the lowest cycle metric and declares it as a decimated frame (not for lack of searching...).

tritical · 15th November 2006, 18:47

@Livesms
It really depends on the clip at hand and how long you are willing to let it take. I rarely have true interlaced material to process (I mainly work with anime), and primarily use tdeint(mode=2) w/ eedi2 for postprocessing. For true interlaced material I think people concerned the most with speed tend to use LeakKernelDeint() and those concerned the most with quality tend to use mvbob() or now mcbob(). TDeint and TDeint+eedi2 are somewhere in the middle.

@ChiDragon
You should be able to do it by changing this code in void Cycle::setDecimateLow(int num) inside cycle.cpp:

Code:

	
for (v=0, i=0; v<num-ovrDec; ++i) 
{
	if (decimate[lowest[i]] != 1)
	{
		decimate[lowest[i]] = 1;
		u = lowest[i];
		while (decimate2[u] == 1) ++u;
		decimate2[u] = 1;
		++v;
	}
}

to

Code:

for (v=0, i=0; v<num-ovrDec; ++i) 
{
	if (decimate[lowest[i]] != 1)
	{
		bool update = true;
		for (int c=max(cycleS,lowest[i]-LIM); c<=min(cycleE,lowest[i]+LIM); ++c)
		{
			if (decimate[c] == 1)
			{
				update = false;
				break;
			}
		}
		if (update)
		{
			decimate[lowest[i]] = 1;
			u = lowest[i];
			while (decimate2[u] == 1) ++u;
			decimate2[u] = 1;
			++v;
		}
	}
}

Replace LIM with the limit you want. I can't test it atm, but it should work... just make sure that lim isn't so large that it can't mark at least cycleR number of frames. To make it an actual parameter in tdecimate would take a little more work. You'd need to modify the cycle class, tdecimate class, and the env->addfunction() call in PluginInit.cpp.

The code in tivtc is definitely not easy follow, but that's what happens after two years of hacking a poorly planned initial design to add new functionality

. I've thought about rewriting it from scratch quite often, but have never found the motivation for that much work.

Livesms · 15th November 2006, 19:22

Quote:

Originally Posted by tritical

@Livesms
It really depends on the clip at hand and how long you are willing to let it take. I rarely have true interlaced material to process (I mainly work with anime), and primarily use tdeint(mode=2) w/ eedi2 for postprocessing. For true interlaced material I think people concerned the most with speed tend to use LeakKernelDeint() and those concerned the most with quality tend to use mvbob() or now mcbob(). TDeint and TDeint+eedi2 are somewhere in the middle.

And what is the most common variant for deinterlacing using EEDI2+TDeint?

MeGui provide sample

Code:

edeintted = last.AssumeTFF().SeparateFields().SelectEven().EEDI2(field=-1)
TDeint(order=1,full=false,edeint=edeintted)

WorBry use such script

Code:

interp = separatefields().eedi2(field=-2)
tdeint(mode=1,order=1,edeint=interp)

and

Code:

interp = separatefields().eedi2(field=3)
tdeint(mode=1,order=1,edeint=interp, AP=5, APType=0 )

Chainmax suggest

Code:

#For TFF Clips: 
Interp = SeparateFields().EEDI2(field=3)
TDeint(mode=1,order=1,edeint=Interp)

and

Code:

#For BFF Clips:
Interp = SeparateFields().EEDI2(field=2)
TDeint(mode=1,order=0,edeint=Interp)

Another one

Code:

Interp = AssumeBFF().SeparateFields().EEDI2(field=3)
TDeint( mode=1, order=1, edeint=Interp, mthreshL=1, mthreshC=2 )

I just messed up

canuckerfan · 16th November 2006, 01:34

Quote:

Originally Posted by tritical

@canuckerfan
Sorry I didn't reply to your last post earlier, I must of missed it. You're correct on TDeint not working correctly in that script if the field order changes. I was thinking of tfm by itself (its pping doesn't depend on field order). If it is deinterlacing only a few frames it'll probably be fine though. I don't know of any automatic way to determine where the field order changes. It's interesting that you get field order changes without dgindex finding illegal tff flag transitions in the stream... what version of dgindex are you using?

i'm using dgindex 1.4.9b3. well, it outputted two d2v files, one with a bad tag and one without the tag. the log for operation stated this:

Code:

D2V Fix Output

Field order transition: 0 -> 2
d00 1 0 2732797456 0 0 b0 b0 90 b0 b0 a0 b0 b0 a0 b0 b0 a0 b0 b0 a0
d00 1 0 2733054080 0 0 b2 b2 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
corrected...
d00 1 0 2732797456 0 0 b0 b0 90 b0 b0 a0 b0 b0 a0 b0 b0 a0 b0 b0 a1
d00 1 0 2733054080 0 0 b2 b2 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2

so i'm guessing dgindex somehow "fixed" the FO?

ChiDragon · 16th November 2006, 04:29

That worked great tritical, thanks!

canuckerfan, if I'm not mistaken you should be able to treat the whole video as either TFF or BFF (whichever the D2V says) since DGIndex fixed the field order switch.

canuckerfan · 17th November 2006, 04:04

^I see. so with the following ivtc method, with the above situation:

Code:

AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=4,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

TDeint's param's need not be changed? And I'm guessing mode 4 or 5 for TFM would still work the best? Thanks.

Mug Funky · 17th November 2006, 04:42

@ tritical:

i just discovered the new compiles of smartdecimate et al.

excellent!

does this solve all the caching problems? because i've always had probs with the old smartdecimate (crash on 2nd pass start the most annoying one), but i just quickly hacked in into my auto-pal function and it worked a treat. massive speedup. and thanks to vinverse plugin no artefacts either!

foxyshadis · 17th November 2006, 10:31

Quote:

Originally Posted by canuckerfan

^I see. so with the following ivtc method, with the above situation:

Code:

AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=4,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

TDeint's param's need not be changed? And I'm guessing mode 4 or 5 for TFM would still work the best? Thanks.

When using dgdecode, overriding field order is superfluous, even dangerous. At the very least annoying needing two different variations, when one does fine.

Code:

Interp = SeparateFields().SelectEven().EEDI2(field=-1)
Deinted=TDeint(edeint=Interp)
TFM(mode=4,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

Tritical, I'm curious, what is the logic behind setting TDeint's field to 0 if hints=true? I'd never noticed that before.

11th November 2006, 05:06	#801 \| Link
tritical Registered User Join Date: Dec 2003 Location: MO, US Posts: 999	What are the numerator and denominator values for the fps of the clip being fed into tdecimate (should be able to see the values using info())? Unfortunately, the mode 2 debug output doesn't tell much about what might be happening except that it isn't getting all the way through tdecimate's constructor. For mode 2 w/ 50 fps input and 29.970 rate, it should spit out: [4548] TDecimate: v0.9.12.5 by tritical [4548] mode2_num = 1 mode2_den = 3 numCycles = 19 clength = 15 [4548] mode2_cfs 0 = 15 [4548] TDecimate: inframe = 0 useframe = 1 rate = 29.969999 If you remove everything after tdecimate from the script does it still crash? EDIT: I was able to run this script without any problems: mpeg2source("C:\test clips\t4.d2v",idct=7,info=3) assumefps(25,1) ColorMatrix(hints=true,interlaced=true) SecureBob() TDecimate(mode=2,input="test.txt",rate=29.970,debug=true) and the same script without stats input. Last edited by tritical; 11th November 2006 at 05:31.

13th November 2006, 02:02	#804 \| Link
canuckerfan Registered User Join Date: Jul 2005 Posts: 317	I see. So if I had TIVTC+EEDI2 called like this: Code: AssumeTFF() Interp = SeparateFields().SelectEven().EEDI2(field=1) Deinted=TDeint(order=1,field=1,edeint=Interp) TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted) TDecimate(mode=1) Changing the modes wouldn't simply work, would it? TDeint params would have to change, right? Wondering if there's any quick way of finding out where the field order changes in the footage without manually scanning through DGIndex. EDIT: well, initially I was using mode 6, and it left some combing in some scenes. however, mode 4 seems to leave almost none. so far, things seem fine without adjusting the TDeint params. Last edited by canuckerfan; 13th November 2006 at 02:22.

13th November 2006, 04:24	#805 \| Link
ChiDragon Registered User Join Date: Sep 2005 Location: Vancouver Posts: 600	tritical: I've got a video that seems to require decimation by 184-frame cycles. Since the cycles are so large, some frames are incorrectly decimated (particularly at fades to black, where it will drop a bunch of black frames and leave duplicates in the real video). Anyway, I was trying to override these parts and TDecimate appears to be ignoring the override line past a certain point. It will only read this far: Code: 0,1000 -++++-++++-+++++-++++-++++-++++-+++++-++++-++++-++++-+++++-++++-++++-+++ Adding anything to the end of this line doesn't actually change the decimation, whether it's + or -. The 72nd frame is where it stops following the override line.

13th November 2006, 23:24	#809 \| Link
ChiDragon Registered User Join Date: Sep 2005 Location: Vancouver Posts: 600	I tried increasing the fgets value from 80 to 300 myself, and now it's actually crashing when the ovr line is longer than 103 chars. Last edited by ChiDragon; 14th November 2006 at 00:08.

15th November 2006, 12:50	#813 \| Link
ChiDragon Registered User Join Date: Sep 2005 Location: Vancouver Posts: 600	tritical, would it be possible to add a parameter such that no 2 dups may be closer to each other than "n" (when cycleR > 1)? I think this would solve the problem I'm having with multiple drops at black frames, and dropping the wrong frames in general with a large cycle of this kind. I've had to do a lot of overrides so far since it's often dropping stationary new frames instead of the actual dups. I know with this video that there should never be two real dups less than 5 frames apart. I was gonna attempt this for myself so I didn't have to bug you again, but I couldn't even find the specific code that takes the lowest cycle metric and declares it as a decimated frame (not for lack of searching...). Last edited by ChiDragon; 15th November 2006 at 12:53.

12th November 2006, 20:21	#802 \| Link
canuckerfan Registered User Join Date: Jul 2005 Posts: 317	i have some dvd telecined footage with an altering field structure (most of the movie is TFF, but sometimes it will change to BFF, I know, it's bad). Is there any possibility of handling the TTF part of the footage as TFF, and the same with respect to BFF. Thanks.

13th November 2006, 00:58	#803 \| Link
tritical Registered User Join Date: Dec 2003 Location: MO, US Posts: 999	You could use mode=4 or mode=5 in tfm, both of which can handle alternating field order since they use 3-way (p/c/n) matching. The alternative is to manually find the places where the field order changes and then either use a series of trim()'s to create a new clip out of each and field match them separately or keep it as one clip and use an overrides file in tfm to change the value for order over the necessary frame ranges.

13th November 2006, 04:52	#806 \| Link
tritical Registered User Join Date: Dec 2003 Location: MO, US Posts: 999	I probably used an 80 byte buffer for reading each line so the rest gets cut off. I'll increase the size in the next release.

14th November 2006, 09:35	#810 \| Link
tritical Registered User Join Date: Dec 2003 Location: MO, US Posts: 999	@canuckerfan Sorry I didn't reply to your last post earlier, I must of missed it. You're correct on TDeint not working correctly in that script if the field order changes. I was thinking of tfm by itself (its pping doesn't depend on field order). If it is deinterlacing only a few frames it'll probably be fine though. I don't know of any automatic way to determine where the field order changes. It's interesting that you get field order changes without dgindex finding illegal tff flag transitions in the stream... what version of dgindex are you using? @Livesms The first two scripts are logically equivalent and should run at the same speed and produce the same output. The third script is the same as the first two except for the full=false part (it will check each input frame to see if it's combed or not). @ChiDragon That's strange that it crashes. I changed all the fgets() calls in tdecimate.cpp/tfm.cpp/tfmpp.cpp from 80 to 1024 and increased the linein[] arrays to 1024 bytes, and everything seemed to work fine. I did a short test with cycle=120, cycleR=2, and the ovr file marking frames 96/116 to drop and it worked fine. What are the cycle/cycleR values you are using? and what are the lines in the ovr file?

14th November 2006, 12:33	#811 \| Link
ChiDragon Registered User Join Date: Sep 2005 Location: Vancouver Posts: 600	Whoops, it never occurred to me to check the linein array declaration! I was busy checking over all the ovrArray stuff... As you may have guessed, I'm not actually a programmer. :P Changing that fixed it of course, thanks. Weird that it managed to work up until the 104th byte though, when it was only set for 80...

15th November 2006, 18:47	#814 \| Link
tritical Registered User Join Date: Dec 2003 Location: MO, US Posts: 999	@Livesms It really depends on the clip at hand and how long you are willing to let it take. I rarely have true interlaced material to process (I mainly work with anime), and primarily use tdeint(mode=2) w/ eedi2 for postprocessing. For true interlaced material I think people concerned the most with speed tend to use LeakKernelDeint() and those concerned the most with quality tend to use mvbob() or now mcbob(). TDeint and TDeint+eedi2 are somewhere in the middle. @ChiDragon You should be able to do it by changing this code in void Cycle::setDecimateLow(int num) inside cycle.cpp: Code: for (v=0, i=0; v<num-ovrDec; ++i) { if (decimate[lowest[i]] != 1) { decimate[lowest[i]] = 1; u = lowest[i]; while (decimate2[u] == 1) ++u; decimate2[u] = 1; ++v; } } to Code: for (v=0, i=0; v<num-ovrDec; ++i) { if (decimate[lowest[i]] != 1) { bool update = true; for (int c=max(cycleS,lowest[i]-LIM); c<=min(cycleE,lowest[i]+LIM); ++c) { if (decimate[c] == 1) { update = false; break; } } if (update) { decimate[lowest[i]] = 1; u = lowest[i]; while (decimate2[u] == 1) ++u; decimate2[u] = 1; ++v; } } } Replace LIM with the limit you want. I can't test it atm, but it should work... just make sure that lim isn't so large that it can't mark at least cycleR number of frames. To make it an actual parameter in tdecimate would take a little more work. You'd need to modify the cycle class, tdecimate class, and the env->addfunction() call in PluginInit.cpp. The code in tivtc is definitely not easy follow, but that's what happens after two years of hacking a poorly planned initial design to add new functionality . I've thought about rewriting it from scratch quite often, but have never found the motivation for that much work. Last edited by tritical; 15th November 2006 at 18:56.

16th November 2006, 04:29	#817 \| Link
ChiDragon Registered User Join Date: Sep 2005 Location: Vancouver Posts: 600	That worked great tritical, thanks! canuckerfan, if I'm not mistaken you should be able to treat the whole video as either TFF or BFF (whichever the D2V says) since DGIndex fixed the field order switch. Last edited by ChiDragon; 16th November 2006 at 04:31.

17th November 2006, 04:04	#818 \| Link
canuckerfan Registered User Join Date: Jul 2005 Posts: 317	^I see. so with the following ivtc method, with the above situation: Code: AssumeTFF() Interp = SeparateFields().SelectEven().EEDI2(field=1) Deinted=TDeint(order=1,field=1,edeint=Interp) TFM(mode=4,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted) TDecimate(mode=1) TDeint's param's need not be changed? And I'm guessing mode 4 or 5 for TFM would still work the best? Thanks.

17th November 2006, 04:42	#819 \| Link
Mug Funky interlace this! Join Date: Jun 2003 Location: i'm in ur transfers, addin noise Posts: 4,555	@ tritical: i just discovered the new compiles of smartdecimate et al. excellent! does this solve all the caching problems? because i've always had probs with the old smartdecimate (crash on 2nd pass start the most annoying one), but i just quickly hacked in into my auto-pal function and it worked a treat. massive speedup. and thanks to vinverse plugin no artefacts either! __________________ sucking the life out of your videos since 2004

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode