Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st August 2005, 03:39   #261  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Good to see that TIVTC is still being worked on . If you need a trouble clip for testing, you could download my Simpsons clip. DGIndex reports it as NTSC, mostly interlaced with a few progressive spots. I have a couple of questions: first of all I'd like to know the difference between mode6 and mode3 (or match on combed and match (same order) on combed, for that matter). Also, I am going to try the following line on an encode:

Code:
TFM(d2v="X:\wherever\myd2v.d2v",mode=3,PP=7,slow=2,mChroma=true,chroma=true,mi=50)
TDecimate(mode=1)
That uses the default flags=1, but I'm using DeDot (dotcrawl remover) before TIVTC, that shouldn't screw up the flag switch operation, right? Also, if I understood correctly, the flags switch assumes that the source was correctly flagged, what could happen if it isn't and the flags switch is used?


P.S: if you download the Simpsons clip, could you confirm if the windows frames on the scene where Homer parks the car move up and down?
Chainmax is offline   Reply With Quote
Old 1st August 2005, 04:05   #262  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
@ tritical:

Quote:
One thing I didn't mention before was that when it does matching off the flags it also disables the post-processing check and just assumes the match is not combed to prevent bad post-processing decisions.
this would be a reasonable assumption... any encoder that wrongly flags the stream with RFF etc would be giving glitchy output anyway (occasional jitters), and i wouldn't consider it TFM's job to fix this, as i've never heard of it happening (though theoretically it could happen on a 2-pass hardware encode when the playback deck glitches on the 1st pass but not the second, or other weird things like that. to be honest i don't know at what point the 3:2 detection happens in hardware encoders, but i imagine it happens on both passes).
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 1st August 2005, 07:46   #263  |  Link
BangoO
Registered User
 
Join Date: May 2004
Posts: 94
Quote:
Originally Posted by tritical
@BangoO
It should behave exactly the same just going off what you gave as info. In the sequence "0 1 2 3 0 1 3 1 2 3" tfm would do field matching off the flags for the 0 1 2 3 0 1 then use its own comparisons for two frames where the fields from the 3 are and then go back to using the flags. TDecimate would get that info, but like I described before it will fall back into its own processing when it encounters more than one d2v marked duplicate in a cycle or on other strange scenarios.

Back in v0.9.7.2 of TIVTC (January 8, 2005) nothing was implemented as far as using d2v info, all it did was check for illegal transitions and the field order. You can get that same behavior with the current version by settings "flags=3" in tfm(). "flags=0" sets the behavior to what it has been doing since v0.9.7 of tfm (february 19th, 2005) up to now. That is check for illegal transitions/field order as well as pass rff info on to tdecimate to help duplicate/hybrid detection.

The current behavior with "flags=1" (which is the default)... is that it does all that flags="0" does plus in 0123 trf flag sections it does the field matching based off the flags. One thing I didn't mention before was that when it does matching off the flags it also disables the post-processing check and just assumes the match is not combed to prevent bad post-processing decisions. Atm this is not completely finalized as it needs more testing to see how well it works on a larger number of sources.
Ok thx for the explanations, I'll give it a try...
I'll try TDecimate as well, I never used it as it was a lot slower than Decimate from the Decomb filters...
Is it a problem if I keep on using Decimate instead of TDecimate (if it is still faster) ?

Last edited by BangoO; 1st August 2005 at 08:14.
BangoO is offline   Reply With Quote
Old 1st August 2005, 08:36   #264  |  Link
BangoO
Registered User
 
Join Date: May 2004
Posts: 94
I'm having surprising speed results...
I encoded 1000 frames with the following configurations:
Option 1: TFM 0.9.9.5 + Decimate
Option 2: TFM 0.9.7.2 + Decimate
Option 3: TFM 0.9.9.5 + TDecimate

The script was the following:
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\mpeg2dec3.dll")
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\TIVTC\TIVTC.dll")
mpeg2source("G:\pouet.d2v")
cropbottom(8)
TFM()
Decimate() # or TDecimate() for option 3
LanczosResize(1280,720)


Here are the results...
Option 1: TFM 0.9.9.5 + Decimate -> 226s
Option 2: TFM 0.9.7.2 + Decimate -> 220s
Option 3: TFM 0.9.9.5 + TDecimate -> 273s

The source is a 100% FILM source, so TFM 0.9.9.5 should have been faster than TFM 0.9.7.2, but maybe I used wrong parameters (all defaults)...
Moreover, TDecimate is still a lot slower than Decimate.

PS: I didn't compare the results, but the size of the 3 result files are exactly the same (19 869 696 bytes) so one can assume the result is the same...
BangoO is offline   Reply With Quote
Old 1st August 2005, 10:03   #265  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
@BangoO
For tfm to do matching off the flags and get a speed benefit you have to specify the d2v parameter... tfm(d2v=""). If you don't then there will be no difference speed wise between tfm from v0.9.7.2 and v0.9.9.5.

The reason TDecimate is slower than decimate is because of the cropbottom() you have there. It prevents the isse code from being used due to the height not being mod 16 anymore so the c code is used. The c code for tdecimate will be slower than decimate due to the extra processing it does. The isse code is definitely faster than decimate by a good bit (2x or more).
tritical is offline   Reply With Quote
Old 1st August 2005, 10:11   #266  |  Link
BangoO
Registered User
 
Join Date: May 2004
Posts: 94
Quote:
Originally Posted by tritical
@BangoO
For tfm to do matching off the flags and get a speed benefit you have to specify the d2v parameter... tfm(d2v=""). If you don't then there will be no difference speed wise between tfm from v0.9.7.2 and v0.9.9.5.
Ok i'll try again then

Quote:
Originally Posted by tritical
The reason TDecimate is slower than decimate is because of the cropbottom() you have there. It prevents the isse code from being used due to the height not being mod 16 anymore so the c code is used. The c code for tdecimate will be slower than decimate due to the extra processing it does. The isse code is definitely faster than decimate by a good bit (2x or more).
Ah nice, I'll definitely try again

Last edited by BangoO; 1st August 2005 at 10:14.
BangoO is offline   Reply With Quote
Old 1st August 2005, 10:19   #267  |  Link
BangoO
Registered User
 
Join Date: May 2004
Posts: 94
And here are the results

Option 1: TFM 0.9.9.5 + Decimate -> 146s
Option 2: TFM 0.9.7.2 + Decimate -> 220s
Option 3: TFM 0.9.9.5 + TDecimate -> 110s

PS: the following script has been used:
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\mpeg2dec3.dll")
Loadplugin("C:\Program Files\AVIsynth 2.5\plugins\TIVTC\TIVTC.dll")
mpeg2source("G:\pouet.d2v")
TFM(d2v="G:\pouet.d2v")
Decimate() # or TDecimate() for option 3
cropbottom(8)
LanczosResize(1280,720)

Last edited by BangoO; 1st August 2005 at 10:21.
BangoO is offline   Reply With Quote
Old 1st August 2005, 10:21   #268  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
@Mug Funky
Quote:
this would be a reasonable assumption... any encoder that wrongly flags the stream with RFF etc would be giving glitchy output anyway (occasional jitters), and i wouldn't consider it TFM's job to fix this, as i've never heard of it happening (though theoretically it could happen on a 2-pass hardware encode when the playback deck glitches on the 1st pass but not the second, or other weird things like that. to be honest i don't know at what point the 3:2 detection happens in hardware encoders, but i imagine it happens on both passes).
That is good to know. The whole matching off the flags rests on the assumption that the encoded frames in the mpeg stream are progressive. That seems highly likely when there is a 0123 pattern, but if there have been long breaks in the pattern somewhere earlier there is the possibility for strangeness. For example, I once saw a stream that had some sections with 0123 and other long sections of just 2222... in the middle of the stream there was a short section looked like this (where it transitioned from a run of 2's into 0123):

x a b c d <= mpeg frame
a b c d e

0 1 2 3 0 <= trf flags

So it was flagged with 0123, but the actual frames were offset by a field... in such a case the matching off flags would fail. I think that was from an hdtv capture that had bitstream errors from transmission or something cause it also had many illegal field order transitions so it is probably not a normal occurence.
tritical is offline   Reply With Quote
Old 1st August 2005, 10:23   #269  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
@BangoO
That is better . The results were still the same hopefully?
tritical is offline   Reply With Quote
Old 1st August 2005, 10:25   #270  |  Link
BangoO
Registered User
 
Join Date: May 2004
Posts: 94
Seems so... the file sizes are still exactly the same, and I also checked some frames, it seems identical.
I will have to test it on a bad flagged material now...

Thx for your great work tritical

Last edited by BangoO; 1st August 2005 at 10:28.
BangoO is offline   Reply With Quote
Old 1st August 2005, 11:11   #271  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
@Chainmax
Quote:
I have a couple of questions: first of all I'd like to know the difference between mode6 and mode3 (or match on combed and match (same order) on combed, for that matter).
The difference between mode 3 and mode 6 is only the order in which the extra matches are tried if the original two candidates are combed. First, a little explanation about the same order thing...

Assume you have a tff clip. Based on the field order the only matches that should work are p/c if you match off the top field or c/n if you match off the bottom field. So if field=1 in tfm (matching off top field) then p/c should be the best choices... if both result in combed frames, the next choice that has the best chance of making a good frame is u (u off top field == n off bottom field). So match same order simply means that it will try the match that makes since field order wise first when the main two matches are combed. Mode 3 doesn't go for the field order wise match first, but goes for the only remaining match that would still match off the same field (either p or n, whichever wasn't tried before). Both strategies have advantages and disadvantages. The u match has a more likely chance of being a good match, but it will create a duplicate because if u works it means the next frame is a p match. The n or p match is less likely to be a clean match (actually, it will only work if there is no motion), but it wont create a duplicate in a scene with motion. For example, in a lot of anime there are pulldown pattern changes at almost every scenechange due to edits and that also means that at quite a few scenechanges there is an orphaned field. In most circumstances, using the same order on combed mode will find a good match but it will create a duplicate... while the n or p match won't do any good at all. That difference is kind of moot in modes 3 and 6 since both end up trying all 5 matches if no good frame is found.

I think the best explanation for the modes if from the readme:

0 = (p/c)
1 = (p/c + n)
2 = (p/c + u)
3 = (p/c + n + u/b)
4 = (p/c/n)
5 = (p/c/n + u/b)
6 = (p/c + u + n + b)

a "/" means it will compare the matches to see which one is best, a "+" means if the previous matches are detected as combed then it will try the following match(es) and if the follwing match(es) are good then it will take it else it will drop back and use the best of the original two matches. One thing that has to be known is that tfm is only capable of comparing two matches if they are both being matched off the same field... i.e. it can compare p to c to n and u to b, but it can't compare u to p/c/n or b to p/c/n and vice versa. So to walk through modes 3 and 6:

mode 6 compares p to c and checks the best match to see if it is combed. If it isn't then it takes it else it creates the u match and checks it. If u isn't combed then it takes it, else it creates the n match and checks it. If n isn't combed then it takes it else it creates the b match and checks it... takes b if it is good else it drops back uses the best between p/c and runs post-processing on it.

mode 3 compares p to c and checks the best match to see if it is combed. If it isn't then it takes it else it creates the n match and checks it. If n isn't comed then it takes it, else it compares u to b and then checks the best match to see if it is combed... if it isn't then it uses it else it drops back and uses the best between the original two matches (p and c) and runs post-processing on it.
Quote:
That uses the default flags=1, but I'm using DeDot (dotcrawl remover) before TIVTC, that shouldn't screw up the flag switch operation, right? Also, if I understood correctly, the flags switch assumes that the source was correctly flagged, what could happen if it isn't and the flags switch is used?
Using dedot before is fine. The only problem that could happen with flags=1 is if you have a very strange stream that has 0123 flagging on non progressive frames. That situation should be rather rare as the whole point of using rff flags in that way is so that only the original progressive frames have to be stored.

Last edited by tritical; 1st August 2005 at 11:18.
tritical is offline   Reply With Quote
Old 1st August 2005, 11:20   #272  |  Link
DarkNite
Almost Silent Member
 
DarkNite's Avatar
 
Join Date: Jun 2002
Location: Purgatory
Posts: 272
After that explanation I'm just itching to see how mode 6 handles some of the more interesting scenechanges in older anime captures that have been sitting in my "waiting for manual edit" bin.
__________________
Rethinking the "Why?" chromosome.
DarkNite is offline   Reply With Quote
Old 1st August 2005, 14:52   #273  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I don't know if this will be useful, but here are two screenshots from a specific frame (both use TIVTC v0.9.9.5):

Script 1:
Code:
MPEG2Source("X:\wherever\SimpTest.d2v",cpu2="ooooxx")
DeDot()
TFM(d2v="X:\wherever\SimpTest.d2v",mode=3,PP=7,mChroma=true,chroma=true,mi=50)
TDecimate(mode=1)
FixChromaBleeding()
LumaYV12(lumoff=-2,lumgain=1.0)
ColorYUV(levels="pc->tv")
FunkyDeblock()
VagueDenoiser(nsteps=6,chromaT=0)
HQDering()
aWarpSharp(depth=16,cm=1)
Crop(12,0,700,476,align=true)
BicubicResize(640,480,0,0.5)
BlindPP(quant=0,cpu=0,cpu2="ooooxx")
LimitedSharpen()
Result:
Link Removed



Script 2:
Code:
MPEG2Source("X:\wherever\SimpTest.d2v")
DeDot()
TFM(d2v="X:\wherever\SimpTest.d2v",mode=6,PP=7,slow=2,mChroma=true,chroma=true)
TDecimate(mode=1)
FixChromaBleeding()
FunkyDeblock()
HQDering()
Crop(12,0,700,476,align=true)
BicubicResize(640,480,0,0.5)
Unfilter(-20,-20)
LimitedSharpen()
Result:
Link Removed

Last edited by Chainmax; 2nd August 2005 at 21:17.
Chainmax is offline   Reply With Quote
Old 2nd August 2005, 19:06   #274  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Hm, it is hard to gather anything from that without the debug or display output or preferably both. One thing that matters when trying the extra matches is the combed frame detection is not very sensitive to really small areas of combing... so if it tries n and it has only a tiny bit of combing it might use it while match u could be completely clean. That is the real tradeoff to trying matches in different orders... the first ones tested have a higher chance of being used.

One option that should be added soon is to only allow u/b matches at scenechanges, which should work well with the orphaned field situation I described above.
tritical is offline   Reply With Quote
Old 2nd August 2005, 20:34   #275  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I'll upload these screenshots again, this time with the debug and display output info, but would those explain the missing star in the second screen?
Also, could you please confirm wether TIVTC causes the issue I mentioned a couple of posts ago?
Chainmax is offline   Reply With Quote
Old 3rd August 2005, 02:08   #276  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
The star now appears with the second script . I must have screwed up somewhere.
Chainmax is offline   Reply With Quote
Old 3rd August 2005, 02:18   #277  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Hm, if using mode=1 in tdecimate you'll have to run straight through the clip to get to that point cause mode 1 results can be different when seeking. Since it is only tfm that differs would it be possible to eliminate the rest of the script for the moment? Determining what exactly happened to cause the difference in the full script would probably require the tdecimate debug output as well.

I suspect the reason the star is gone is just because tfm used a different match in the two different modes, but the debug output will tell for sure.

I downloaded the clip, but didn't notice any window moving when he parks the car. You are talking about the part right around frame 450 correct?
tritical is offline   Reply With Quote
Old 3rd August 2005, 02:34   #278  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I was using the "Go to" option, so it must have been the seeking issue then, because the window now appears. Thanks for the reply.

The problem in the indows should be somewhere between frames 320 and 400 after IVTC. The source doesn't display such problems. The problem was mainly caused by aWarpSharp, but I think in this case TIVTC might have contributed a bit which is why I'm asking you to double-check for me since I'm not 100% sure after several tests.
Chainmax is offline   Reply With Quote
Old 4th August 2005, 22:25   #279  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
If it did I am either too blind to see it or am not looking for the right thing cause I don't see anything.
tritical is offline   Reply With Quote
Old 6th August 2005, 06:51   #280  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Do you know those times when you have a computer related issue and when you try to show it to someone else it goes away? That's what happened to (yet another time) here. First the "missing star", now this . I feel really stupid, sorry for wasting your time like this .


P.S: I didn't notice any difference at first glance when using the flags switch, but then again I'm not too perceptive at that sort of things. What were your impressions on how d2v flag based matching handled the clip?

Last edited by Chainmax; 6th August 2005 at 06:56.
Chainmax is offline   Reply With Quote
Reply

Tags
tdeint, tivtc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:12.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.