Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd November 2004, 00:43   #81  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
So, TFM and TDecimate will be separate filters? Also, once you finish TFM and TDecimate, what will happen to TDeint (i.: will you strip its fieldmatching and deinterlacing capabilities)?

[edit]Nevermind, I just read the readme.

Last edited by Chainmax; 22nd November 2004 at 00:51.
Chainmax is offline   Reply With Quote
Old 22nd November 2004, 04:18   #82  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Yes, they will be separate filters, but will both be in the same dll (not sure what the dll will be called yet). I don't have plans to remove anything from TDeint atm, but will probably disable trying to weave by default. TDeint still needs to have a new edge-directed deinterlacing method added and gonna add the ela method from tomsmocomp as well since a lot of people seem to prefer it. I was never able to come up with a useable post-processing fix to what xappy posted earlier (one that didn't also touch most of the frame), and pretty much consider those type of artifacts as inherent to motion adaptive deinterlacing.
tritical is offline   Reply With Quote
Old 22nd November 2004, 18:24   #83  |  Link
AS
Registered User
 
Join Date: Jan 2002
Posts: 76
Running risk of being off-topic, I would like to request a feature for Tdecimate: being able to choose a fixed numbered duplicate to drop; say should tdecimate detect a row of duplicate, I can selectively drop every 2nd duplicate, rather than always dropping the 1st one in a fixed cycle, which is what decimate mode=0, mode=2 does.
AS is offline   Reply With Quote
Old 22nd November 2004, 21:52   #84  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Could you explain a little bit more about why that option would be useful? To me it seems like that would go with user overrides (manually specifying a decimation pattern). Also, mode 0 of decimate just drops the most similar frame in each cycle, it doesn't look for duplicates like mode 2 so I'm not sure how it would work in that context.

A couple things about longest string decimation. There is a reason for removing in order from first to last in duplicate strings. It's so that when you calculate the length of duplicate strings in the next cycle you get an accurate count going back into the previous cycle (if any duplicate strings run back into the previous cycle). Since the duplicate string tracking goes across 3 cycles: previous, current, and next. But as Leak said, their all duplicates, so there are workarounds possible. Currently what TDecimate does is it removes in order of lowest metric from duplicate strings, but when it calculates the length of the duplicate strings for the next cycle it acts as if it had decimated in first to last order in the previous cycle (which shouldn't make any difference since they are all duplicates). It requires a little more code, but it accomplishes both goals. TDecimate's longest string decimation is already quite a bit more involved then Decimate's since it needs to work for M-in-N decimation and not just 1-in-N.
tritical is offline   Reply With Quote
Old 22nd November 2004, 23:57   #85  |  Link
AS
Registered User
 
Join Date: Jan 2002
Posts: 76
I thank you for the correction of my misunderstanding and the explanation with regard to longest string decimation. I will try with your suggestion of manual override.

Just a clarification, for my personal understanding, if you don't mind.

Quote:
Currently what TDecimate does is it removes in order of lowest metric from duplicate strings...
Do you mean it drops the duplicate with the lowest metric, then the one with the 2nd lowest metric and so on?

If I am correct, then there is a case there you do not want to drop the one with lowest metric first but start with the one of the highest metric. Suppose you have an ordered duplicate set/string of AA', I am not sure about you, but in my experience A' is always cleaner than A; A might have chroma shift and/or key-frame corruption if A is a key-frame; A' has none of that and has less compression artifacts by the nature of lossy video compressions. In this scenario, A would have a higher metric than A', would it not? Should you have a longer duplicate string with this high-metric bad frame somewhere in the string, you would end up keeping the frames you don't want and losing the frames you want.

Unless I have misunderstood it again ^^;
AS is offline   Reply With Quote
Old 23rd November 2004, 03:31   #86  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
to change the subject slightly, would it be possible to include something like a matroska timecodes file after decimation, so that hybrid streams could be left untouched but the advantages of having less frames could be exploited?

i guess this would need 2 passes, as VfW needs to know how many frames it will be dealing with.

what i'm doing right now is going 3 passes with dedup, 1 after the other in a VDM job list - the first is null output (done with ffdshow set to uncompressed and no output) which gathers dup metrics, the second is xvid 1st pass, using the dup metrics to decimate, and the third is xvid 2nd pass.
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 23rd November 2004, 04:40   #87  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
@Mug_Funky
If your talking about doing vfr with mkv by decimating frames in film portions and leaving 30p alone and then outputing the needed matroska timecodes file I have been planning that since the beginning. The blending of 30p to 24p for a single framerate method of dealing with 30p I didn't initially plan to add, since I use matroska almost exculsively with the vfr method I just described, but it is added now as well. If your talking about also adjusting the timecodes file to remove all frames with metrics below a specific threshold like dedup then no, I am not currently planning to do that. I had originally planned the mkv vfr stuff to be two-pass, but a one-pass version based off Leak's earlier suggestion will probably also be added.

@AS
You are correct about how it works. Your suggestion that A' would be a cleaner frame then A in an AA' dup scenerio is also something to consider. However, doing what you said for duplicate strings would pretty much eliminate the benefit of doing it in the lowest to highest order... which was that you will still get correct decimation in cases where frames are incorrectly declared as dups when they aren't due to low metrics (this happens on very slow panning scenes among others). I guess things could just be changed to decimate the frame before the frame with the low metric (both in most similar and in longest string decimation) instead of the low metric frame itself. For the moment I'm not going to do it, but I will put it on my things to test list.

@Leak/All
I was thinking more about the one-pass mkv vfr mode (with addition of plain black frames onto the end to pad as needed and including the actual ending frame number as a comment in the timecodes file), and aside from the consecutive cycle limit needing to be 2 or 1, it would also require completely linear access from start to finish to work correctly. If these requirements aren't too much to deal with I will add it.

Hybrid handling through blending of 30p sections will also be able to be done in 1 pass with the requirement that the required number of consecutive cycles is limited to 2 or 1 as in the mkv vfr 1 pass case. The blending one pass does not have the requirement for completely linear access though and will work correctly when seeking.
tritical is offline   Reply With Quote
Old 23rd November 2004, 09:07   #88  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
that's music to my ears, tritical

i wonder if using avs processing inside of ffdshow would be a way to overcome the problem of needing the final frame count? i suppose directshow needs the frame count as well.
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 25th November 2004, 11:57   #89  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I am testing TFM(mode=2) and noticed slight jaggyness on some frames. After reading the readme, will altering any of the parameters eliminate this issue?

BTW, would there be any chance to replace blend for interpolation as a deinterlacing method?
Chainmax is offline   Reply With Quote
Old 25th November 2004, 20:23   #90  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
It depends on whether or not the jaggyness is caused by deinterlacing or bad matching. I can't tell whether or not you meant it is from deinterlacing or not in your post, if so just ignore this next part. Could you run TFM with "display=true" and see wether the bad frames have been detected as combed and deinterlaced or are just bad matches? After that it'll be easier to find out whats wrong.

There is already a non-blending option in tfm, its the interpolation from tdeint... but you could use the "clip2" parameter and basically get any type of deinterlacing you want. If you want cubic interpolation try something like this:

mpeg2source("c:\mysource.d2v")
deinted = last.bob().seleteven() #assuming tff source
tfm(clip2=deinted)

You can expand that to use deinterlacing from another filter or whatever. When using clip2 you do need to make sure that the field that is kept by the deinterlacer is the same as the field that tfm is matching from. See the clip2 description in the help file of tfm for more info. If many people think tfm should have linear or cubic interpolation built in I guess I could add it, but its not currently on my todo list.
tritical is offline   Reply With Quote
Old 25th November 2004, 21:56   #91  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I'll try the display option soon and report back.
Chainmax is offline   Reply With Quote
Old 28th November 2004, 02:55   #92  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Alright, well here is [link removed, look farther into the thread]. Yeah, I have no imagination for names. It includes TFM and TDecimate. I've tried to test everything to make sure it all works, but there may still be a couple things that slipped by. The options for TDecimate are not well organized and a little chaotic atm, but it gets the job done. The only thing that is not implemented is hybrid handling for cycleR > 1 (cycleR is the M in M-in-N decimation), all hybrid handling is currently limited to cycleR = 1 only. I have a plan for how hybrid handling for cycleR > 1 should work, but it would take a while and I'm not sure that there is even any need for it. Mainly just need some willing testers and feedback about what needs to be changed. The zip includes a "common usage examples" file that shows the syntax to use for most of the basic cases.

The one pass vfr w/ mkv timecode output can be done with the following script for example:

mpeg2source("c:\mysource.d2v")
tfm(d2v="c:\mysource.d2v")
tdecimate(mode=3,mkvOut="timecodes.txt")

The d2v option in the above script was added to TFM, among a few other fixes/changes, it scans the d2v file for illegal field order transitions (such as Fixd2v or DGFix) and will create a fixed version if anything wrong is found, and also will determine the field order from it if order is set to -1. The d2v option will work with d2vs from dvd2avi 1.76 and variants, dvd2avidg, and dgindex.

I'm also interested in feedback about the file input/output options if anyone uses two pass mode. Currently the filters do not check if a file with the same name already exists for output files (if there already is one it is overwritten), and for input files neither filter does any internal checking to make sure the file goes with the current video (such as calculating a crc for the first few frames).

EDIT: Decided to add the crc checking to output/input files. Now, whenever an output file is created TFM/TDecimate create a crc32 value based off the first 15 frames of the clip and store the value in the output file. On loading a file for input the value from the file is tested to make sure it is the same as the value for the current clip.

Also added a "batch" parameter to TDecimate that will disable some checks and memset two arrays to fake values allowing for a script with tdecimate(mode=5, ...) to be loaded when the input and tfmIn files have no entries. This allows you to set up vdub's job control with both scripts (a first pass script and a second pass script) at the same time and run them, meaning you don't have to be there when the first script/pass finishes.

Last edited by tritical; 28th November 2004 at 21:05.
tritical is offline   Reply With Quote
Old 28th November 2004, 08:32   #93  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
this looks very very cool. i can't test it right now unfortunately, but i will as soon as i can (provided the computer isn't too hot... just coming into summer here).

good work, as always
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 28th November 2004, 18:33   #94  |  Link
igor1st
Registered User
 
Join Date: May 2004
Posts: 57
TFM v0.9.3 crashs when mode=3.
igor1st is offline   Reply With Quote
Old 28th November 2004, 21:04   #95  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Sorry about that, forgot to adjust the conditions of an array allocation for the PP parameter when it was added. That was causing modes 0 and 3 of TFM to crash if PP was greater then 0. Here's TIVTC v0.9.2, it fixes it.
tritical is offline   Reply With Quote
Old 29th November 2004, 04:20   #96  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
nice! i finally got past 30000 frames on Lain and there was no crash. perhaps the illegal field-order stuff in the d2v was the culprit?

at defaults i still got 1 combed mouth in the ~30 minutes of footage passed through it. not bad.

question: does Tdecimate treat static scenes as 30p or 23.976 when set to output mkv timecodes? there seemed a lot of bits at 30p in the test i did, even though the whole thing (except the intro and one or two special effects) is a hard-telecine. perhaps i should mess with the thresholds some?
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 29th November 2004, 05:47   #97  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
It depends on the vidDetect setting. You can read about it in the readme, but it can be set to detect video sections either off matches, metrics, matches or metrics, or matches and metrics. At default it is set to the last option, metrics and matches. Which means that for a cycle to be detected as video both the matches used by TFM indicate there are not duplicates and the metrics calculated by TDecimate are all above vidThresh. The other option that controls video detection is conCycle, even if the above requirements of vidDetect are met for one cycle the conCycle parameter sets a minimum consecutive cycle limit before any cycle will be marked as video. On default settings it should detect static sections as 23.976 film since all frames need to have metrics above vidThresh... unless the vidThresh value is too low for the clip. The metric parameters (dupThresh and vidThresh) i set relatively low and are geared toward pretty clean material at defaults. You might try using display=true in mode 4 and then go to a static section of the clip to set a more appropriate value for your specific clip. Or you could do it in two pass mode and set a higher conCycleTP value, by defaults conCycle is set to 2 which is its maximum and conCycleTP also defaults to 2.

Also could I bug you for a little sample containing the mouth movement that wasn't matched correctly?
tritical is offline   Reply With Quote
Old 29th November 2004, 07:46   #98  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
ach! i deleted it to make room for some urgent DVD copying. i had 20 gigs worth of backup on crap discs, and i have to get them copied onto good discs before they die completely... there's one spinning in the only drive that will read it now, and windows says there's 17.8 million minutes remaining... that's my priority at the moment, but i'll try to find that bit again when my data's safe
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 30th November 2004, 18:12   #99  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
@Mug_Funky
If you already deleted it don't worry about, you sound like you have enough to do already .

@All
Here is [link removed, look farther down the thread] nothing big, changes:
Code:
TDecimate:
     - fixed mishandling of the last cycle of a clip when in mode 3
         and the last cycle was film
     + write blockx and blocky sizes into tdecimate metric log files
     + timecode v2 format output support

TFM:
     + significant speed up (30-50%) to modes 1, 2, and 4 with 
         PP > 0, due to some missing logic the output frame would
         needlessly get checked twice for combing

Last edited by tritical; 8th December 2004 at 05:05.
tritical is offline   Reply With Quote
Old 30th November 2004, 20:35   #100  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I was going to use the following lines in an encode:

X=MPEG2Source("wherever\somesource.d2v")
Y=X.Trim(1,0)
ConditionalFilter(X,X,Y,"IsCombed","<","14")


and then wondered if the IsCombed parameter could be replaced with IsCombedTIVTC so that I'd have one less plugin to load on the script. I know that both functions hould be the same, but since IsCombed only has one parameter and IsCombedTIVTC has more I'm not sure if what I'm asking can be done.
Chainmax is offline   Reply With Quote
Reply

Tags
tdeint, tivtc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:24.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.