Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th June 2005, 01:59   #241  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,607
mean, how will tdecimate know which frames are near duplicates and not simply different frames with a small amount of motion?

Tdecimate already knows, that x, d AND n are duplicates of each other, because I only see one of them in the final output.

So blending the most un-similar duplicates should be possible, because the string of duplicates (here xdn) is already known..
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 5th June 2005, 02:28   #242  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
That is only true under perfect conditions, which is definitely not always true. There is the possibility that there are more non-duplicates in a given sequence then there should be given a specific decimation ratio. In the star trek voyager sequence you sent me a while ago there were a few places where this was the case. That means one of the dropped frames could actually be a non-duplicate and blending it will produce ghosting. Also, you want the blending only when there are triples? Or blending for all dups?
tritical is offline   Reply With Quote
Old 5th June 2005, 02:57   #243  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,607
That means one of the dropped frames could actually be a non-duplicate and blending it will produce ghosting.

But it also will reduce jerkyness a bit, if tdecimate is unsure about which frame to choose.

Also, you want the blending only when there are triples? Or blending for all dups?

No.

I had this idea: blend all most different duplicates of one row.

examples of duplicate chains (non-real-world):
n1,n2,n3 -> slight different near/new duplicates
d -> bit identical, or better said: almost identical duplicate of the preceding frame

this x d n1 will blended to: x+n1 OR d+n1
this x d n1 d1 will blended to: x+n1 OR x+d1 OR d+n1 OR d+d1
this x d n1 n2 n3 will blended to: x+n1+n2+n3 OR d+n1+n2+n3

Most duplicates after R24-processing are bit-identical. So a near-duplicate(s) should be detectable...
Of course, this assumtion does not fit for other sources,
as well as for the ST-Yoyager sample I sent to you because I compressed that one, so the duplicates arent identical anymore.


I hope I didn't cause too much confusion.
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 5th June 2005, 23:27   #244  |  Link
MOmonster
Registered User
 
Join Date: May 2005
Location: Germany
Posts: 495
@scharfis_brain
Nice idea. Reduce jerkiness by blending the near doubled frames. But wouldnīt it be better just to choose the frame with better quality, because the blending will also produce a little softness in many cases (in some sources I tested some time ago). Also the reducing of the jerkiness wouldnīt be so strong just by blending the frames together.
In most cases the second duplicated frame is the frame with the better quality. Another point is that the match with less combing also seems to have the better quality. Maybe the decimater could read the informations from the matcher, that he could decide what frame to choose. But as I know restore24 use telecide to match the fields, donīt know if something like this is possible. But in sources I tested last time it would be better just to choose the right frame than blending them together.
Itīs just an idea.

@tritical
I can understand you. We will see, what is going on in the future. I have begun learning C and C++, maybe in the future I will make my own filters. We will see.

Last edited by MOmonster; 5th June 2005 at 23:29.
MOmonster is offline   Reply With Quote
Old 7th June 2005, 02:52   #245  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Well, here is [link removed] ... it adds the new matching mode (new mode is mode 2, old modes starting at mode 2 have been bumped up one so old mode 2 is now mode 3 and so on). It also adds support for the new dgindex d2v files (v9). Gonna see about adding a general purpose combing measure output filter based on the 5 point metric. Currently tfm can output the match statistics it calculates via outputdebugstring() if debug=true, but those statistics are match specific (i.e. can't be compared to statistics from other matches).

@scharfis_brain
So I'm still not exactly clear on how tdecimate will know which frames are near duplicates. You're saying to look for two bit identical (or very close) frames and then consider all frames after and up to the next pair of identical frames to be near duplicates?

Last edited by tritical; 26th June 2005 at 04:45.
tritical is offline   Reply With Quote
Old 15th June 2005, 01:13   #246  |  Link
MOmonster
Registered User
 
Join Date: May 2005
Location: Germany
Posts: 495
Hi tritical,
it took some time, but some minutes ago I finished my function. I tried to create a function, that realize my idea of a blenddetection and here it is.
Code:
LoadPlugin("C:\program_me\dgmpgdec\DGDecode.dll")
LoadPlugin("C:\program_me\filters\TIVTC.dll")
LoadPlugin("C:\program_me\filters\FDecimate.dll")
#LoadPlugin("C:\program_me\filters\TDeint.dll")
LoadPlugin("C:\program_me\filters\leakkerneldeint.dll")

#AVISource("D:\video\inu\[Ingow] Pirates Of Japan (Trailer).avi")
#OpenDMLSource("C:\Dokumente und Einstellungen\Benutzername\Eigene Dateien\Eigene Videos\film.avi")
#DirectShowSource("D:\video\inu\[Ingow] Pirates Of Japan (Trailer).avi")

MPEG2Source("D:\dvd.d2v",cpu=4,iPP=false)


#trim(0,0)

crop(8,8,-8,-8)


a = Cdeint()

tfm(order=1, mode=0, display=false, slow=false, pp=2, clip2=a)

fdecimate(rate=23.976,threshold=1.0,metrics=false) 	# or another decimater


function Cdeint(clip clp) 
{

###### PREPARATION ######
global sourceclip = clp.leakkernelbob(order=1, sharp=false, twoway=false, threshold=7, map = false)
global testclip = sourceclip.duplicateframe(0)

###### FRAMES FOR BLENDPARAMETERS ######
global combing_n0 = testclip.selecteven()
global combing_n1 = testclip.selectodd()
global combing_n2 = combing_n0.trim(1,0)
global combing_n3 = combing_n1.trim(1,0)

###### POSSIBLE OUTPUTS ######
out = sourceclip.selecteven()
out_1 = sourceclip.selectodd()

###### Conditional Function Chain, evaluated from bottom to top (!) ######
c99=ConditionalFilter(clp, out_1, out, "btest/Min(combl_0, combl_1)", "lessthan", "btest_2/Min(combl_1, combl_2)") 

c9=FrameEvaluate(c99, "global btest_2 = LumaDifference(blendmade_test, blendpic_test)")
c8=FrameEvaluate(c9, "Evaluate_test()")
c7=FrameEvaluate(c8, "global blendl_test = combl_1 < combl_2 ? 0.5 * Exp(0.66 * Log(combl_1 / combl_2)) :
	\	1 - 0.5 * Exp(0.66 * Log(combl_2 / combl_1))")
c6=FrameEvaluate(c7, "global btest = LumaDifference(blendmade, blendpic)")
c5=FrameEvaluate(c6, "Evaluate()")
c4=FrameEvaluate(c5, "global blendl = combl_0 < combl_1 ? 0.5 * Exp(0.66 * Log(combl_0 / combl_1)) :
	\	1 - 0.5 * Exp(0.66 * Log(combl_1 / combl_0))")	
c3=FrameEvaluate(c4, "global combl_2 = LumaDifference(combing_n2, combing_n3)")		
c2=FrameEvaluate(c3, "global combl_1 = LumaDifference(combing_n1, combing_n2)")      
c1=FrameEvaluate(c2, "global combl_0 = LumaDifference(combing_n0, combing_n1)")      
return(c1) 
}


# (running inside of the Conditional Environment)

function Evaluate() 
{
vid = testclip.selecteven()
vid_1 = vid.trim(1,0)

global blendmade = MergeLuma(vid,vid_1,blendl)
global blendpic = testclip.selectodd()
}

function Evaluate_test() 
{
vid_2 = testclip.selectodd()
vid_3 = vid_2.trim(1,0)

global blendmade_test = MergeLuma(vid_2,vid_3,blendl_test)
global blendpic_test = testclip.selecteven().trim(1,0)
}

function Min(float a, float b)
{
temp = a < b ? a :b
return(temp)
}
Yes, I know there are still many tweaks possible. Till now, I donīt use a special mode to save more clear fields (the mode I explained some posts ago), there is no noise parameter till now, we canīt change the bobber and the most important point is, that the used values are still tweakable.
But also this early version seems to work really good.
I just post it here, that you could have a look on it. We spoke about it some posts ago.

Last edited by MOmonster; 15th June 2005 at 15:16.
MOmonster is offline   Reply With Quote
Old 23rd June 2005, 18:58   #247  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I am trying to rip one of my DVDs, but my current script crashed VDubMod (or Avisynth, don't know). After checking filter by filter, I discovered that TFM was causing the crash. Any idea why? Here's the script:

Code:
MPEG2Source("X:\wherever\Test.d2v",cpu2="ooooxx")
DeDot()
TFM(d2v="X:\wherever\Test.d2v",mode=3,PP=7,mChroma=true,chroma=true,mi=50)
TDecimate(mode=1)
FixChromaBleeding()
LumaYV12(lumoff=-2,lumgain=1.0)
ColorYUV(levels="pc->tv")
HQDering()
Crop(12,0,700,476,align=true)
BicubicResize(640,480,0,0.5)
BlindPP(quant=0,cpu=0,cpu2="ooooxx")
LimitedSharpen()
Chainmax is offline   Reply With Quote
Old 23rd June 2005, 20:46   #248  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
I'm gonna guess that you are using a very recent version of dgindex? If you open your .d2v file in notepad and look at the first line what is the version number (DGIndexProjectFilexx, the xx)? If it is 10, then I haven't gotten around to releasing a version with support for that yet, and tfm will throw an error when it finds that it isn't a supported d2v version. Actually, I added support for v10 a while ago (only needed to change 1 line), but also added a few new features to tivtc and haven't been able to force my self to update the docs so it can be released.

On a side note, when you say crashed... does that mean vdubmod disappears without an error msg? If you're on xp sp2 and using a 2.5.6 beta, try replacing the avisynth.dll in your system directory with the one from this zip (save a copy of your current one if you want to revert back), it should correctly report the error. If it is indeed a problem with supporting the d2v version.

Last edited by tritical; 23rd June 2005 at 20:51.
tritical is offline   Reply With Quote
Old 24th June 2005, 19:12   #249  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Yes, I'm using the latest DGIndex and it uses v10. VDubMod does disappear without an error message and I am using an updated WinXP and Avisynth v2.5.6b3. I will try the dll, even though it's kinda evident from your post that it's a problem with DGIndex v10 since I really miss the error messages.


[edit] do I only have to replace the avisynth.dll, or the directshowsource.dll and TCPDeliver.dll files too?

Last edited by Chainmax; 24th June 2005 at 19:15.
Chainmax is offline   Reply With Quote
Old 24th June 2005, 22:59   #250  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Just avisynth.dll should do the trick. Nothing of significance has changed in tcpdeliver since beta 3, and the only change in directshowsource was a fix for rmvb getting stuck at 100%.
tritical is offline   Reply With Quote
Old 25th June 2005, 00:19   #251  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
You were right, it's a DGIndex problem .
Chainmax is offline   Reply With Quote
Old 25th June 2005, 00:38   #252  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
Quote:
Originally Posted by Chainmax
You were right, it's a DGIndex problem .
Is there really a problem with DGIndex, or are you just trying to combine incompatible tool versions?
Guest is offline   Reply With Quote
Old 25th June 2005, 05:14   #253  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I was agreeing with him, and he didn't mention that DGIndex itself has an issue. I just phrased myself wrongly, sorry about that.
Chainmax is offline   Reply With Quote
Old 26th June 2005, 04:45   #254  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Here is [link removed], it supports dgindex v10 d2v files.

Last edited by tritical; 31st July 2005 at 09:30.
tritical is offline   Reply With Quote
Old 28th June 2005, 18:45   #255  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Works like a charm .
Chainmax is offline   Reply With Quote
Old 31st July 2005, 09:30   #256  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
Unfortunately, development on TIVTC/TDeint has slowed a lot due to my being busy and working on other projects, but I did have some free time earlier today so here is [link removed].

This version adds a new field matching mode to tfm (mode 6, so it doesn't change any previous mode numbers), and the ability for tfm to field match off the d2v flags (can be only in 0123, a.k.a film to dvd2avi/dgindex, sections or completely). With the addition of the d2v matching to tfm, tfm/tdecimate can now do ivtc completely off the flags in properly flagged sections. The decimation is not 100% strict to allow for hopefully better handling. How it works is tfm passes on the duplicate info, which is figured by taking into account the rff flags and the actual matches used, to tdecimate. TDecimate then checks that that frame's metric is indeed <= to the lowest metric in the cycle (which it should be) and that there is only 1 d2v duplicate in the cycle. If those conditions aren't met then it uses it regular handling. Field matching off the flags gives a very large speedup since no calculations have to be done.

Also, I finally went through and changed all the setcachehints calls for all filters in tivtc.dll to use a diameter instead of radius... so depending on script complexity it could give a pretty good speed up for most people.

Any feedback on the d2v flag based matching would be helpful, especially if it doesn't work correctly. I tested it a good bit, but the only material I have atm is either 100% film or 0.0% film, nothing that is split. I also have been doing some work on improving the field matching algorithm, and would be very grateful for sample clips that have matching problems (ideally where a good match exists between the normal matches - p/c or n/c).

Full change list:
Code:
TFM:  Added new match mode (mode 6)
TFM:  Added field matching from d2v file
TFM:  Added "flags" parameter to control how d2v info is used or not used
FieldDiff:  can use sse instead of sad
ALL:  setcachehints diameter instead of radius

Last edited by tritical; 1st August 2005 at 00:00.
tritical is offline   Reply With Quote
Old 31st July 2005, 09:32   #257  |  Link
BangoO
Registered User
 
Join Date: May 2004
Posts: 94
tritical, what's the advantage of matching off d2v flags, only speed ?
How much faster is it ?

Thx
BangoO is offline   Reply With Quote
Old 31st July 2005, 09:42   #258  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
The advantage is speed and also not risking incorrect matches due to mistakes by tfm's matching algorithm. The same reasons why it is recommended to use force film vs using an auto ivtc method when dvd2avi/dgindex report 100% film or a very high percentage film. In my tests on the movie castaway tfm ran about 6 to 7 times faster than when it had to actually calculate matches. In terms of raw numbers it jumped from about 130-150 fps to 1000-1100 fps. In cases where dvd2avi/dgindex report 100% film there is still no advantage to using tivtc, but in cases where it is <100 you get the benefit of flagged ivtc in most of it (assuming it all works correctly of course ) and intelligent handling in the rest. TIVTC will still be slower than force film since tdecimate still does metric calculations, but tdecimate is extremely fast now for material that is mod 16 width/height since the isse optimized code for that path was added.

Last edited by tritical; 31st July 2005 at 09:48.
tritical is offline   Reply With Quote
Old 31st July 2005, 11:17   #259  |  Link
BangoO
Registered User
 
Join Date: May 2004
Posts: 94
Ok tritical, I'll give it a try then.
I'm just a bit "scared" to change my encoding method as the tfm I use is just so perfect right now... (TIVTC v0.9.7.2).

My concern is on sources that are supposed to be 99% "0 1 2 3" but which contain some drops in this sequence most of time caused by bad recording (something like "0 1 2 3 0 1 3 1 2 3", that's only an example).
In these cases, TIVTC v0.9.7.2 is just perfect... what about the new one ?

Last edited by BangoO; 31st July 2005 at 11:20.
BangoO is offline   Reply With Quote
Old 31st July 2005, 23:53   #260  |  Link
tritical
Registered User
 
Join Date: Dec 2003
Location: MO, US
Posts: 999
@All
Another new version: [link removed]. It adds a second slower matching fuction. The "slow" parameter is now an int instead of bool. The default is 1 so it uses the same matching function as previous versions with slow=true. Slow=2 may perform better in some cases where slow=1 fails.

@BangoO
It should behave exactly the same just going off what you gave as info. In the sequence "0 1 2 3 0 1 3 1 2 3" tfm would do field matching off the flags for the 0 1 2 3 0 1 then use its own comparisons for two frames where the fields from the 3 are and then go back to using the flags. TDecimate would get that info, but like I described before it will fall back into its own processing when it encounters more than one d2v marked duplicate in a cycle or on other strange scenarios.

Back in v0.9.7.2 of TIVTC (January 8, 2005) nothing was implemented as far as using d2v info, all it did was check for illegal transitions and the field order. You can get that same behavior with the current version by settings "flags=3" in tfm(). "flags=0" sets the behavior to what it has been doing since v0.9.7 of tfm (february 19th, 2005) up to now. That is check for illegal transitions/field order as well as pass rff info on to tdecimate to help duplicate/hybrid detection.

The current behavior with "flags=1" (which is the default)... is that it does all that flags="0" does plus in 0123 trf flag sections it does the field matching based off the flags. One thing I didn't mention before was that when it does matching off the flags it also disables the post-processing check and just assumes the match is not combed to prevent bad post-processing decisions. Atm this is not completely finalized as it needs more testing to see how well it works on a larger number of sources.

Last edited by tritical; 14th August 2005 at 08:47.
tritical is offline   Reply With Quote
Reply

Tags
tdeint, tivtc

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:22.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.