TDeint and TIVTC

[Adub]

It's a little dated, but check this out:
http://avisynth.org/mediawiki/VFR#encoding_to_vfr_.28mkv.29

[thetoof]

Yeah, I read it... but I thought I had done something wrong, because nothing was being written in the "tfm.txt" and "stats.txt".

I've tried running a preview, an analysis pass and a lossless pass in vdub without getting anything... any ideas why it doesn't work?

And, do you know the format of the timecode?

Thx

[Adub]

You did let the passes finish fully, correct? Because no information is written until the script is done. I was doing the same thing and I realized that I had to wait until the end before the info was written.

as for the timecodes, I would have to look at them to tell you. Or you could do it yourself: Check which of these the file looks like:
http://www.bunkus.org/videotools/mkvtoolnix/doc/mkvmerge.html#EXTERNAL%20TIMECODE%20FILES

[thetoof]

That's exactly what I needed, thanks! Everything's working perfectly fine now.

[Adub]

Glad to help.

[shadell]

I have tried using TDeint with the blend interpolation option as a replacement for fielddeinterlace(), which I find produces pleasing results in some situations, despite the ghosting. The call was TDeint(type=4). This produced a lot of jagged artefacts, such as the following:

http://i297.photobucket.com/albums/mm223/shadelll/artifact013crop.png

When I look at the map=2 output, I see that the effected areas have (mostly) not been marked as 'interpolate pixel'. When I look at the interlaced input, the area is combed.

Reducing mthreshL and mthreshC down to 1 reduces the number of artefacts, but there are still quite a few.

Since there is a blend interpolation option, I thought it should work OK. Are there other options I need to use to get it to work without producing these artefacts?

[Nikos]

a. The denoise option of TDecimate is for internal metric calculations only or not?

denoise -

Sets whether or not to denoise frames prior to doing difference metric calculations.
This can greatly improve metrics for noisy sources (i.e. stabilize duplicate frame
metrics and make it easier to tell the difference between duplicates and non-duplicates).
It also works very well for sources with lots of dot-crawl because the denoising
effectively cancels all dot-crawl, whereas the dot-crawl would usually inflate difference
values of duplicates. Using denoising will slow things down somewhat, but it is MMX
optimized and pretty fast.

true - use denoising
false - don't


b. May i crop before IVTC with tfm and TDecimate?

mpeg2source("Black&White-source.d2v")
Greyscale()
crop(32,16,-32,-24)
tfm(d2v="source.d2v", mode=1, mChroma=false)
Tdecimate(chroma=false)


c. The greyscale place is right?
The source is Black&White telecine 29.97 fps film.

[Adub]

Since your clip is already black and white, why add the greyscale parameter? Greyscale() is usually used when converting color clips to greyscale.

Also, I believe that it is never good to crop before an ivtc, or whenever dealing with interlaced material. I thought I heard there was a way some time ago, but I have always found it safer to crop afterward.

[Nikos]

Thanks Merlin7777 for the answer.
Althought the clip is black and white, sometimes there is noise in the uv planes.
For interlaced or telecined material i know that it's safe to resize horizontal before IVTC but i am interesting if it's safe to crop before IVTC or deinterlace.

[Adub]

Althought the clip is black and white, sometimes there is noise in the uv planes.

Ah, I understand then.

And like I said before, cropping before IVTC or Deinterlacing is never a safe thing to do. I have never gotten it to work correctly at least.

[Boulder]

I'd expect that cropping horizontally is OK, and cropping vertically mod-4 also to be OK.

[Mystery Keeper]

I'd expect that cropping horizontally is OK, and cropping vertically mod-4 also to be OK.

Only cropping mod16 is safe. One should crop mod16, do what needs to be done and then crop the rest.

[Didée]

Only cropping mod16 is safe. One should crop mod16, do what needs to be done and then crop the rest.
Ah, a statement. Please explain why it should be so.

(I don't see why mod16 should be required for purposes of IVTC or deinterlacing in general. mod4 vertically is a technical requirement for YV12 sources, since that's the "luma distance" between same-parity chroma samples. That's all.)

[Mystery Keeper]

Ah, a statement. Please explain why it should be so.

(I don't see why mod16 should be required for purposes of IVTC or deinterlacing in general. mod4 vertically is a technical requirement for YV12 sources, since that's the "luma distance" between same-parity chroma samples. That's all.)

I tried cropping before deinterlacing with TempGaussMC and it gave me ghosting ^_^'

[Didée]

During testing TGMC, I'm often cropping wildly around on the input, only caring for mod-4, not -16, and never got any unusual ghosting. Feel free to post script + material that makes your observation reproducable.

Again, mod4 is the only technical requirement in this case.

[Adub]

Ah, so that was the trick to it. it's okay to crop horizontally. but when cropping vertically, always respect mod 4.

[thetoof]

Hi,
I've got something fairly complicated to do via vfr decimation, so I've got a help request if it's doable and a feature request if it isn't :p

Let's say I have these framerates :
a=23.976
b=25
c=29.97
d=59.94
They were then spliced together with
a.changefps(d)+b.changefps(d)+c.changefps(d)+d
So, I have a CFR clip @ 59.94fps with tons of duplicates (that I want to remove via vfr decimation). How can I do that???

Also, there are many duplicates in the 23.976 clip and it could be decimated to very low framerates for a couple of frames (even below 8 fps) So, I was wondering if it would be possible to have a vfr decimation that adapts to the amount of motion and uses framerates other than the standards ones to display the same motion to encode a lot less frames with no visual difference (in anime you can even have the same frame for more than 1 second after all the noise has been removed).

Here is a test clip I have created for the occasion that replicates the situation I just described: http://rapidshare.com/files/141496007/VFR_testclip.7z.html

[sander815]

how do i call the multithreaded version of this plugin? EEDI2_imp.dll

[foxyshadis]

You replace EEDI2.dll with that one. That's it.

[aand]

Question about TDecimate:
I have a series of black frames; instead of cutting some of them, it removes all of them and leaves more 'visible' dups behind.
Is there a way of setting <max nr of dups in a row> ? If not, I'm making an official feature request :D.
Thanks.

[r00t61]

Is there any way I can set the location of the debug output information provided by TFM/TDecimate (i.e, when display=true)? Currently, the output is always fixed to the upper left-hand corner, so if I have display=true on for both TFM/TDecimate, the debug output from TDecimate overlays on top, obscuring the debug output for TFM.

I'd like to be able to see both sets of debug data at the same time.

[thetoof]

I didn't check the doc, but a quick and dirty way of doing it is by using stackvertical/horizontal to see the same frame with different info, or by cropping the part of the frame where the info is to overlay it on another corner.
source=your source
tfminfo=source.tfm(settings).crop(select the part of the frame containing the info here)
tdecimateinfo=source.tdecimate(settings)
overlay(tdecimateinfo,tfminfo,coordinates where you want to overlay)
#uncomment the next line to tweak the crop settings
#return tfminfo

[r00t61]

Well, thetoof, your suggestion didn't quite work out the way I wanted to, but I'll keep experimenting. Thanks.

In the meantime, is there any to alter TDecimate's behavior for vfr output (i.e., when mode=5), for handling 30p video sections? In particular, I would like to make TDecimate perform a bob (59.94 fps) instead of a blend decimation for those sections designated as video, while still IVTCing/decimating the film portions to 23.976 fps.

[thetoof]

You may be interested in AnimeIVTC, as it bobs interlaced sections & IVTC telecined ones to create a VFR clip. See mode=3, 4 or 6 with omode=2.

[canuckerfan]

i have a clip here which i'm trying to deinterlace correctly... it's hard telecined without any blends so an ivtc should work. however, I'm having a hard time determining the field order. I've tried the separatefields().assumetff test but both tff and bff show no backward motion. weird. so far I've thrown 2 scripts at it:

interp = nnedi(field=1)
deint = tdeint(mode=0,order=1,field=1,edeint=interp,slow=2,emask=TMM(mode=0,order=1,field=1))
tfm(mode=3,order=1,clip2=deint,slow=2)
tdecimate()

interp = nnedi(field=1)
deint = yadifmod(edeint=interp)
tfm(mode=3,order=1,clip2=deint,slow=2)
tdecimate()

both produce pretty much the same result to my eyes and the ivtc looks pretty decent. but I'm wondering if this is right with the field order discrepancy. here's the clip(23 mb): http://www.sendspace.com/file/upa8m7

[AVIL]

IMHO your clip is progressive. No temporal difference between fields. Only spatial one. There is some residual combing that you can fight with plugin vinverse. Internal function Info() says that the clip is bff .In fact, when assumed as bff, after separatefields() the first field has the top line. But deinterlace this progressive clip is unnecesary (again IMHO). In scenes with motion there is blending. Try to deblend first.

[canuckerfan]

IMHO your clip is progressive. No temporal difference between fields. Only spatial one. There is some residual combing that you can fight with plugin vinverse. Internal function Info() says that the clip is bff .In fact, when assumed as bff, after separatefields() the first field has the top line. But deinterlace this progressive clip is unnecesary (again IMHO). In scenes with motion there is blending. Try to deblend first.
thanks for your input. why i seem to think it's partially interlaced is because of the horizontal lines evident throughout the clip. classic sign of interlacing. in fact, it seems to follow a 3p/2i pattern. as for blending... i separated the fields and didn't see any myself. during high motion there is plenty of blurriness, however.

[manono]

AssumeTFF().SeparateFields()#plays smoothly
AssumeBFF().SeparateFields()#plays jerky

The sample is TFF and any simple IVTC seems to work. It's not field-blended, although there does seem to be some sort of motion blur or similar going on.

[canuckerfan]

AssumeTFF().SeparateFields()#plays smoothly
AssumeBFF().SeparateFields()#plays jerky

The sample is TFF and any simple IVTC seems to work. It's not field-blended, although there does seem to be some sort of motion blur or similar going on.
yea, I don't know what the hell that stuff is. thanks. your input has confirmed my thoughts.

[AVIL]

I have repeated my test but using dgindex/dgdecode. Formerly I've used directshowsource (an enourmous mistake). My new results agree totally with manono's ones.

[@Tron@]

Interested in a plan to update the gorgeous plug -TFM!? I take them a long time and would like to know whether in the near future, its updating and some "delicious"))) because there is no limit to perfection, and should always be striving towards the ideal ))

And what with TDecimate and MergeHints???

[ikarad]

TIVTC is multithread or not?

[tritical]

@ikarad
No, tivtc isn't multithreaded.

@@Tron@
Atm, I don't have any plans to update tivtc further. My original motivation for coding/working on tivtc was the fact that I actually used it regularly, but as I haven't encoded anything in 2-3 years that motivation is pretty much gone. Now I tend to stick to filters that interest me academically.

[rebkell]

Is there anyway to get tdecimate to ignore the bottom part of the clip, when making decisions on which frames to drop, all the animated logos, and popups on all the network shows that I capture really plays havoc with trying to get the right frames dropped.

[tritical]

I think 'clip2' parameter of tdecimate can accomplish what you want. Something like:

saved = last
crop()
tdecimate(clip2=saved)

It will make decisions based on the cropped clip, but return frames from clip2. If you only crop the bottom of frames it shouldn't effect hint passing.

[rebkell]

I think 'clip2' parameter of tdecimate can accomplish what you want. Something like:

saved = last
crop()
tdecimate(clip2=saved)

It will make decisions based on the cropped clip, but return frames from clip2. If you only crop the bottom of frames it shouldn't effect hint passing.

Thanks, I'll give it a try.

[rebkell]

I think 'clip2' parameter of tdecimate can accomplish what you want. Something like:

saved = last
crop()
tdecimate(clip2=saved)

It will make decisions based on the cropped clip, but return frames from clip2. If you only crop the bottom of frames it shouldn't effect hint passing.

That seems to have worked perfectly. Thanks again.

[ikarad]

@ikarad
No, tivtc isn't multithreaded.

@@Tron@
Atm, I don't have any plans to update tivtc further. My original motivation for coding/working on tivtc was the fact that I actually used it regularly, but as I haven't encoded anything in 2-3 years that motivation is pretty much gone. Now I tend to stick to filters that interest me academically.

thanks for your answer.
Is it possible to hope a multithreading version of TIVTC or is it an utopia?

[tritical]

Is it possible to hope a multithreading version of TIVTC or is it an utopia?
It is very unlikely. With the way the code is, it would be a major task to multithread tivtc interally.

[ikarad]

I have a problem with TIVTC and ffdshow.
If I use TIVTC with libavcodec (MPeg 2 codec), I d'ont have 24 fps but 28 fps.
If I use libmpeg2 instead of libavcodec, I have 24 fps

with libmepg2
http://nsa07.casimages.com/img/2009/05/10/mini_090510040236829569.jpg (http://www.casimages.com/img.php?i=090510040236829569.jpg)

with libavcodec
http://nsa08.casimages.com/img/2009/05/10/mini_090510040442784974.jpg (http://www.casimages.com/img.php?i=090510040442784974.jpg)


without TIVTC
http://nsa08.casimages.com/img/2009/05/10/mini_090510040600831936.jpg (http://www.casimages.com/img.php?i=090510040600831936.jpg)


If I use decomb there isn't any problem with libavcodec or libmpeg2

I use MPC-HC and ffdshow-mt 2940

It is very unlikely. With the way the code is, it would be a major task to multithread tivtc interally.

The problem is that with blu-ray movie at 30 fps, even my cpu Q6600@3.0ghz isn't enough powerfull. I must use Decomb but Decomb is less good than TIVTC

[tritical]

Some options that could speed up processing are:

tfm:
slow = 0
micmatching = 0 or 2

tdecimate:
chroma = false

Of course, changing those could change the output, but in most cases it probably wont be noticeable. I would try slow=0 in tfm and chroma=false in tdecimate, and see if it is fast enough.

[PzSniper]

***removed because my crossposting was a rule #8 infringement***

Sorry neuron2 :(

but...

why i keep gettig this error?
http://i36.tinypic.com/2d7de1k.jpg

[DivxBr]

Sorry for a (maybe) stupid question, but when the TFM plugin sends a popup message telling me that it created an xxxx-FIXED.d2v file, to use, this file (FIXED) must be used on TFM command only or in the mpeg2source command too? This FIX increases/decreases the movie's total frame count?

[tritical]

In both. It could increase or decrease the count... it tries to keep the frame count as close as possible to the original while fixing the field order changes. dgmpgdec also has a tool to do this fixing.

[DivxBr]

I tried the Fix d2v option (in DGindex), but it show no errors to correct(!).
My concern is that this increase/decrease or frames could lead to audio synch errors.

[tritical]

Can you post the d2v file? Usually, the change in frame count is only 1 frame so not noticeable. If you are concerned then you could always ignore it, and use a matching mode that can deal with changing field order.

[B.F.]

Can you post the d2v file? Usually, the change in frame count is only 1 frame so not noticeable. If you are concerned then you could always ignore it, and use a matching mode that can deal with changing field order.
Also has this problem.

Here my d2v file.
Hope it help.

[DivxBr]

Can you post the d2v file? Usually, the change in frame count is only 1 frame so not noticeable. If you are concerned then you could always ignore it, and use a matching mode that can deal with changing field order.

I dont have this file anymore (If it helps, I can generate it again). But I checked the corrected file against the uncorrected, and found one difference only in the last frames, so no problem.

My concern, is if there's a possibility with bad encoded material that the frame count difference accumulates (within corrected file), so we get "a lot of" frames of difference?

[thetoof]

Hi,

I can't find how to detect the small 30p vertical motion of this clip to create good VFR timecodes. http://forum.doom9.org/showthread.php?p=1361082#post1361082

Script:
i=source
match=i.tfm(slow=2,clip2=i.tdeint(2,edeint=i.nnedi2(-2),emask=i.tmm(1)))
bob=i.tgmc.selecteven() #use whatever tgmc version you like to test
match.trim(0,454)+bob.trim(455,503)+match.trim(504,0)
TDecimate(4, output="stats.txt",denoise=true)
#TDecimate(5,hybrid=2,input="stats.txt",mkvout="timecodes.txt")

Any help please?

On a side note, it would be nice to have the option of creating a vfr clip that'd mix 23.976, 29.97 and 59.94 for situations like these (where full motion is actually 60p and is mixed with 24p content).

[tritical]

In mode 5 with vidDetect=3 and no tmfIn file, tdecimate assumes all matches are c. So the only reason a cycle (every 5 frame group, so 0-4,5-9,10-14,etc...) wouldn't be marked as 30 fps is if one or more of the frame difference metrics in that cycle is < vidThresh. Thus, try lowering vidthresh (you can look at the debug/display output in mode 4 to find the lowest difference value in that range of frames). I replaced tgmc with bob() for quick testing, but it looks like with denoise=true you need vidThresh=0.8 for that section. However, in a few cycles I can't tell if there is or is not a duplicate... sometimes the difference metrics go down to 0.3x, but those look like dups to be me. Alternatively, you could use manual overrides to force video detection or manually edit the tdecimate stats file (increase the difference values).


On a side note, it would be nice to have the option of creating a vfr clip that'd mix 23.976, 29.97 and 59.94 for situations like these (where full motion is actually 60p and is mixed with 24p content).
I agree :), but at the time I wrote the hybrid handling in tdecimate I was only concerned with mixes of 24p and 30p content.