PDA

View Full Version : Dup 2.0 for Avisynth 2.5


Guest
2nd January 2003, 07:42
Here is Dup 2.0 beta 1 for Avisynth 2.5.

It supports both YV12 and YUY2.

The algorithm was changed so that when there is a string of duplicates the last frame of the string is used as the copy frame (previously the first frame was used).

In deference to the great sh0dan, the copy parameter was defaulted to true. :)

Feedback will be appreciated.

Guest
2nd January 2003, 08:30
Please use this one:

Gant
2nd January 2003, 11:21
Thanks a lot... was waiting for this one to arrive to 2.5x platform :D

I'll test it as soon as possible... prolly tonight. :)

SILICON
2nd January 2003, 13:52
One sugestion for speed your filter.

Test the difference with the previous frame only in luma plane.

Guest
2nd January 2003, 15:17
Originally posted by SILICON
One sugestion for speed your filter.

Test the difference with the previous frame only in luma plane. Been there, done that. See the 'chroma' option.

gamr
2nd January 2003, 17:23
Downloading to toy with it now.

Just a suggestion, make luma=false too perhaps so as under yv12 it can sample a 1/4 res chroma only (insanely fast, 20% of the original data compared to default settings in yv12 if im not mistaken)

FuPP
2nd January 2003, 21:43
Hi Neuron !

I've tried your new version; very pleased to see it working with avisynth 2.5 :)

Two little problems at this time :

1/ get weird frames (one on two) when using show=true

2/ still using show=true, displayed information (when good frames) seems strange to me :

ex : frm 123 : using frm 124 and so on...

Is it normal ?


Regards,

FuPP

Guest
2nd January 2003, 22:40
Originally posted by FuPP
1/ get weird frames (one on two) when using show=trueWhat exactly is a "weird frame"?

2/ still using show=true, displayed information (when good frames) seems strange to me :

ex : frm 123 : using frm 124 and so on...

Is it normal ? That is saying that frame 123 was asked for but the filter determined that 123 and 124 were duplicates, so it used the last duplicate in the string, in this case 124.

FuPP
2nd January 2003, 23:03
1/

Originally posted by neuron2
What exactly is a "weird frame"?


It means that (sorry for bad english):) (see attachment) :


2/ Not sure to totally get you (because I don't think there were duplicated frames a the place this information was displayed), but will make more tests when I will have solved 1/.


Thx for your quick answer,

FuPP.

Guest
2nd January 2003, 23:49
Originally posted by FuPP
2/ Not sure to totally get you (because I don't think there were duplicated frames a the place this information was displayed Realize that the filter is looking ahead at frames you haven't reached yet. It doesn't compare backwards any more, only forwards.

Guest
2nd January 2003, 23:53
Originally posted by FuPP
It means that (sorry for bad english):) (see attachment) :
Ooh, that's the MakeWritable() bug I've been struggling with! sh0dan tells me it is fine, but... I need to look at the Avisynth source code but it is not easy as I have to get a CVS client, blah, blah.

In the short run I'll hack in my personally written MakeWritable2() that I used in Decomb.

Bulletproof
3rd January 2003, 00:39
That's what happens to me with some filters that I use after a crop, maybe if you put it before a crop it won't do that?

Guest
3rd January 2003, 01:06
Here's the fixed version with local MakeWritable().

Please let me know if this fixes your weird frames. Thank you for pointing out this problem, FuPP.

Guest
3rd January 2003, 01:10
Originally posted by Bulletproof
That's what happens to me with some filters that I use after a crop, maybe if you put it before a crop it won't do that? It happens without any cropping. When I replace calls to MakeWritable() (code is in avisynth.dll) with my own routine that performs the same function, the problem goes away. I am still investigating, but it sure looks like a bug in Avisynth 2.5.

Gant
3rd January 2003, 11:02
Now that i was about to say that i found thta "weird" frame problem someone was ahead of me... lol

Well anyway have to try this new version...

And from what i've seen last night its fast... havent made a table with real values, but hope to do it tonight. :)

Thanks for the update.

sh0dan
3rd January 2003, 11:43
Seems like Pitch isn't being respected properly - is this something only happening in newer versions, or has all 2.5 alphas had this bug?

Are your CPU's ISSE capable? - otherwise that could be the root of the problem (a faulty MMX bitblit).
Are you using the newest avisynth.h (from CVS).

sh0dan
3rd January 2003, 12:21
I cannot believe it... I think I found some remains of the days, when YV12 was forced to be mod 16. Anyway - put up a new binary - try that one out, it _may_ solve the problems.

@neuron: Thanks for being so persistent on convincing me there was a bug - I cannot believe it hasn't been found yet - something must have triggered it lately.

HarryM
3rd January 2003, 13:56
Originally posted by sh0dan
I cannot believe it... I think I found some remains of the days, when YV12 was forced to be mod 16. Anyway - put up a new binary - try that one out, it _may_ solve the problems.

@neuron: Thanks for being so persistent on convincing me there was a bug - I cannot believe it hasn't been found yet - something must have triggered it lately.


Repair the link for download, please. It's related to build 17122002 still.

UPollaehne
3rd January 2003, 14:09
@HarryM
in the meantime you can try this: AVISynth 2.5 Alpha (http://cultact-server.novi.dk/kpo/avisynth/avisynth_a_030103.zip)

@shodan
Did you update CVS?

sillKotscha
3rd January 2003, 14:20
Originally posted by sh0dan
Anyway - put up a new binary - try that one out, it _may_ solve the problems.

Guys - you're unbelievable!! Your efforts and reached goals can't be described with my poor english. You work so hard just for your/ our pleasure - I ask myself why the économic situation just sux nowadays with so many genial heads out there?!!

------------------------

just tried to download the 'debugged' new Binary [03-01-2003] but it's still avisynth_a_17.12.02.zip (my explorer cache is empty)

regards Sill

Edit: too late with my post :D

sh0dan
3rd January 2003, 14:30
Link corrected (the experienced hax0r would still try to download avisynth_a_0301003.zip :)

FuPP
3rd January 2003, 21:12
@Donald+Shodan

hem, hem...

Well, thanks a lot for your efforts, but I still have the problem both with avs 2.5 03/01 + dupyv12b2 or with Dupyv12b3.

Here's my script (I could have start with that last time... :rolleyes: )

LoadPlugin("C:\video\avsfilters\MPEG2DEC3yv12.dll")
LoadPlugin("C:\video\avsfilters\dupyv12.dll")
mpeg2source("f:\tests\vts_01.d2v",CPU=6,IDCT=2)
Crop(8,4,704,568)
Dup(threshold=5,maxcopies=10,chroma=false,show=true)
BilinearResize(448,542)
AddBorders(16,17,16,17)


:confused:

FuPP

XP1800+, W2K

FuPP
3rd January 2003, 21:13
Originally posted by Gant
Now that i was about to say that i found thta "weird" frame problem someone was ahead of me... lol

Sorry for that ! ;)

FuPP
3rd January 2003, 22:09
Oh !

I've discovered that putting another filter (ex : convolution3d or... warpsharp ;)) after Dup make things works : it seems very similar to the problem I've reported to Sansgrip about fluxsmooth...

Hope it can help.;)

Guest
4th January 2003, 19:32
@FuPP

Thank you for your results. I was indeed able to duplicate the problem using your script.

Apparently the vi->IsWritable() is not reliable. I've temporarily removed the test of it in MakeWritable2(). The problem seems to go away. Please test with the attached beta 4 and let me know.

@sh0dan

Any comments about vi->IsWritable()?

Guest
4th January 2003, 19:55
Originally posted by sh0dan
Seems like Pitch isn't being respected properly - is this something only happening in newer versions, or has all 2.5 alphas had this bug?Don't know because I've been upgrading faithfully and only run the newest versions.

Are your CPU's ISSE capable? - otherwise that could be the root of the problem (a faulty MMX bitblit). Athlon XP.

Are you using the newest avisynth.h (from CVS). No. Should I be? Why? Can you attach it here, please? :)

FuPP
4th January 2003, 21:01
@neuron

yes yes yes yes yes !

Nice shot ! It works fine now (and is fast !) Thx a lot for your time.

Best regards,
FuPP.

Guest
4th January 2003, 22:03
Originally posted by FuPP
yes yes yes yes yes !

Nice shot ! It works fine now (and is fast !) Thx a lot for your time.
Thank you again, FuPP, for your assistance. I'd better have a look at Decomb. It has the same code. :(

sh0dan
5th January 2003, 10:39
Originally posted by neuron2
No. Should I be? Why? Can you attach it here, please? :)
The latest is on the alpha site :)

Another possible problem could be, that you read pitches _before_ making the image writable. Pitch might change when you make an image writeable. Could that be the cause?

Guest
5th January 2003, 20:05
Thank you, sh0dan. I will get the new header file and check my pitch usage, as you suggest.

Guest
8th January 2003, 19:23
Here is Dup 2.00 beta 6.

It adds a requested feature: blend. When blend=true, the copy frame will be generated from an averaging of all the duplicate frames. This should give some noise reduction. Please read the help file for details. Of course, with blend=true, the filter will be slower.

Now even MarcFD can't argue that Dup is useless. :)

Feedback will be appreciated.

sh0dan
8th January 2003, 20:17
You might want to check out the Fluxsmooth thread - this could be the reason for your crashes.

Guest
8th January 2003, 20:39
You say that it should be fixed BOTH in the resizer and the application! How would we fix it in the application?

Also, I made the problem go away by removing the test for IsWritable() and its return in MakeWritable(), so that it always copies to a new frame. That doesn't seem to fit with the mechanism you are suggesting?

sh0dan
8th January 2003, 20:54
What happends is that your filter returns frames with different pitches.
If you don't make all frames writable, but only some of them, it returns the source frame.

If the source has been cropped right before your filter, it results in a pitch much larger than the image. When you make an image writable, a new frame is created, with a smaller pitch, and the image is copied to this frame. If you make all images writable the new pitch is consistent. But if you on some occations returns the same frame with a old pitch it causes an error in resize.

I've corrected the error in resize.

The easiest way for you to fix it is to create a new frame, and bitblit to it, if there are no changes - otherwise, just make _all_ frames writeable, no matter which frames are requested.

Guest
8th January 2003, 20:59
Thank you for the explanation. I did notice it didn't happen if I removed the resize after Dup.

Can I get the corrected Avisynth to test the fix?

sh0dan
8th January 2003, 21:01
I'll upload a new binary tomorrow - I don't have access to the server right now. Hope you can wait that long!

Guest
8th January 2003, 21:03
No problem, Sir. Thank you for your excellent analysis and debugging on this issue. :D

FuPP
8th January 2003, 21:27
I have tested b6, but only on a short clip (about 1700 frames). I will try to make more serious tests later in the week.

b6 is stable ;)

b6 without blend : gain in compressibility (mpeg2) : 4 %
b6 with blend : gain = 5.5 %

command : Dup(threshold=5,maxcopies=20,chroma=false,blend=true)

Cheers,
FuPP.

MaTTeR
8th January 2003, 21:47
It's a free lunch so I couldn't resist:D

Has anyone been successful using Dup on clean progressive movies? Just curious more than anything.

FuPP
8th January 2003, 23:01
I've made some attempts a few weeks ago (with avs 2.0x compatible version); the filter did not detect subtle movements in still scenes (like a far character walking in the background), the movie got then jumpy.

But seems really efficient on animes !

Regards
FuPP.

MaTTeR
8th January 2003, 23:11
@FuPP

Thx for the reply. My experience is also exactly as you describe, just wanted to verify it.

Ewi
9th January 2003, 22:26
@neuron2

is it possible to let Dup write a log file which frames are replaced so you can easily check with vdub if there are frames that should not be replaced (for example the described situation with a guy walking far away)...

Thank you...

Guest
9th January 2003, 22:41
Originally posted by Ewi
is it possible to let Dup write a log file which frames are replaced so you can easily check with vdub if there are frames that should not be replaced (for example the described situation with a guy walking far away)Just set debug=true and the needed information will be visible through the DebugView utility (available at my web site). Then use the logging capabilities of the DebugView utility if you want to capture the output to a file.

Ewi
9th January 2003, 23:29
@neuron2

Ohhh, Ok thank you.

What about this idea:

I don't know exactly how Dup is working but I as far as I know it's working on whole frame and if two (or more) frames have a difference less than a threshold the second frame is replaced by frame 1 (except for blend...). But in anime there are often parts of the picture that don't move (for example a person speaking and only its mouth is changing).

So what is about partly "dupping" the frame. A simple way would be to divide the picture in submatrices, f.e. 3x5 or more, and use dup on each of them. A more complex method could require to detect motion, chroma and lumi changes and to mask the parts of the frame that are under a given threshold of a weight function f(motion[atPixel],chromachange[atPixel],lumichange[atPixel]), and then dup or blend-dup only these parts.

Both of this would allow to go lower with the threshold values that have to be actually used and so perhaps our man on the horizon is walking again with a gain in compression in his pocket....

Thank you...

P.S.: If Dup is working in this or a better way: don't hate me for asking silly questions....

Guest
10th January 2003, 00:05
@Ewi

Thank you for your suggestion. The idea was already suggested on the main Dup thread. Search for "New duplicate detection filter". I am considering it but fear that it may fragment motion in an ugly way.

@all

I have released Dup 2.00, the first release version. It has several robustness fixes versus the last beta, and fixes a bug such that the blended frame wasn't used when show=true. It's a good idea for all Dup users to get this version. Please get it from my web site (the Avisynth 2.5 version). The source code is also available there.

Thank you all for your valuable feedback during development.

Marc FD
10th January 2003, 18:03
lol, it'ld be a temporalsmoother ^^

>Now even MarcFD can't argue that Dup is useless.

i still argue it's useless.
your work is great, as always, don, it's all the fault of the dumb guy who had the idea ^_^.

are you still using the same algo to detect movement, by searching the biggest mean difference in 32x32 windows ?

because i've a new filter, with a similar approach, and i'd like to know the drawbacks ^^

Guest
10th January 2003, 18:08
Originally posted by Marc FD
are you still using the same algo to detect movement, by searching the biggest mean difference in 32x32 windows?Yes.
because i've a new filter, with a similar approach, and i'd like to know the drawbacks ^^Can't say without knowing the purpose of your filter.

Marc FD
10th January 2003, 18:10
>Can't say without knowing the purpose of your filter.
just a temporal smoother more ^^.

Guest
10th January 2003, 18:12
Originally posted by Marc FD
just a temporal smoother more ^^. Your statement is too brief and cryptic to understand.

Do you mean block-based conditional copying/blending as suggested by Ewi and others?

Marc FD
10th January 2003, 18:16
sorry don.

okay, let's go technical

pixel-based 5x5 (or more) interleaved diff with inloop affine (based on the accumulation of 5x5 diffs) blending.

sh0dan
13th January 2003, 23:37
@neuron: If you're interested, I've written a very fast ISSE routine that returns the largest accumulated sum of differences of a 32x32 pixel area on the entire image.

It processes 16 pixels in parallel, with less than 1 cycle per pixel, so it's probably maxed out at memory speed. It'll be committed to avisynth now, but it's not activated, because I'd like to reference test it, so it would be great if you could help there.

It might actually not be useful for what I'm doing, but it's just an experiment.

cvs sourceforge link (http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/avisynth2/avisynth/focus.cpp?rev=1.3.2.8&content-type=text/vnd.viewcvs-markup) see int TemporalSoften::isse_scenechange(const BYTE* c_plane, const BYTE* tplane, int height, int width, int pitch)

Guest
14th January 2003, 00:58
@sh0dan

Thank you for the offer and link. I need the function to return the x and y coordinates of the block so that I can display the little box when show=true. Any chance of that?

sh0dan
14th January 2003, 08:49
Not entirely impossible - I can send you an offset x + (y*width/32) that should be enough. Pure luck I still have an mmx register, but I may need two to avoid memory reads, or stalls.

edit: cr*p I didn't have an mmx-register - not even a general purpose register. Couldn't you use the C-code, when show is true? I assume the C-code does exatly the same, so if you test the results, and see if they are the same, it would be pretty safe to use it.
It could added to the code, but in real life there isn't anything time-critical, when show=true (ie. it's not an encode).

Edit2: Wouldn't it make more sense, for it to be used as scene-change detector to add all the differences together and simply threshold-test this value.

Edit 3: Just saw, that the code assumes that the pitch of both images is the same - not a good thing. I'll correct it.

Guest
15th January 2003, 18:18
Couldn't you use the C-code, when show is true? I assume the C-code does exatly the same, so if you test the results, and see if they are the same, it would be pretty safe to use it.
It could added to the code, but in real life there isn't anything time-critical, when show=true (ie. it's not an encode).
Yes. That would be acceptable.

Wouldn't it make more sense, for it to be used as scene-change detector to add all the differences together and simply threshold-test this value.Yes, of course. That application is very different from a duplicate detector.

Just saw, that the code assumes that the pitch of both images is the same - not a good thing. I'll correct it.If you fix that I can give it a try.

Thank you.

esby
24th January 2003, 14:16
Good thing,
both protagonists are here ^^ (see below...)

This code giving me avisynth error...

It seems linked to cnr2() and dup() [edit : Cnr2 and not Dnr2, sorry for the typo ]

I put the whole script cause it could come from other plugins used...



source = avisource("...").killaudio()
source = source.SwapUV()

#used to correct somes frames before decomb fails on them...
#note : frames 36 and 37 are KF but frame 36 is just baaaaad (mixing of 35 & 37)
dest = source
dest = dest.freezeFrame(36,37,37)
#(...) some freezeframes too
dest = dest.freezeFrame(36249,36250,36250)
source = dest

#ivcting cause raw is 29.76 fps (anime)
Source = source.Telecide(chroma=true,dthreshold=13)
source = source.Decimate(cycle=5,show=false,mode=2)

#cropping evil baaaaad bar & resizing
source = source.crop(0,2,628,478)
source = source.BiLinearResize(640,480)

#one of the two plugin alone works
#but both together just won't work
source = source.Cnr2("xxx",6,7,255)
source = source.dup()

#just to check the 1000 first frames in nandub ( or vdub)
return source.trim(0,1000)



When i put
dup() before cnr2()
or cnr2() before dup()
...I m getting an "avisynth read error : AviSynth caught an access violation at 0x01651c17, attempting to write frame 0x00000000..."

Any idea why, or how to avoid it, (I tried moving it before ivtc etc.,
but i haven't found a way to do not crash avisynth actually.)
of course i can do cnr in vdub... But that just mean that cnr2 avs is useless for me in this case ;)


Versions I am using:

AViSynth: 2.50 beta 22 jan. 2003
Decomb: Version 4.06
Dup: 2.00 ( took from the realm of Mordor ;) )
Cnr2: Chroma Noise Reducer for AviSynth (v2.4)

esby

Guest
26th January 2003, 15:48
@esby

You say dnr2() in one place and cnr2() in another. I assume you mean cnr2().

I put Dup() first in the script followed by a debug build of Cnr2(). It crashes in Cnr2():

PVideoFrame __stdcall Cnr2::GetFrame(int n, IScriptEnvironment* env)
{
if (n<1) return child->GetFrame(n, env);

PVideoFrame src = child->GetFrame(n, env);

if (lastn+1!=n || !pre) {
pre = child->GetFrame(n-1, env); // CRASHES HERE !!!
env->MakeWritable(&pre);
}

const uc *srcp = src->GetReadPtr();
uc *prep = pre->GetWritePtr();
...It crashes in the Avisynth core on the marked GetFrame() call when stepping from frame 0 to frame 1. I verified that n-1 is 0 at this call.

So it appears to be a bug in the Avisynth core, maybe the cache. We need to get sh0dan involved. I will PM him to draw his attention to this thread.

sh0dan
26th January 2003, 20:13
Couldn't it just be CNR2 crashing?

It has been known to do that.


(but I'll look at it ASAP)

esby
26th January 2003, 20:31
It's possible,
But as far i remember cnr2 alone or dup2 alone worked for the first frame processed.
And both crashed together.
I think i ended removing both, not satisfacted of the final result.
(prolly cause dup blending mode not really good with some 'decombed' frames)

esby

Guest
26th January 2003, 20:56
@sh0dan

I traced through Cnr's code and it crashes on the simple GetFrame() call for frame 0. I don't know how you can blame Cnr2. But by all means if you can think of a mechanism let me know and I'll look some more at it. Thank you.

sh0dan
27th January 2003, 17:21
@neuron: Sorry - I thought you were listing the Dup-code (thus the crash would be in CNR2's GetFrame). I'll do a testsetup, and hopefully find it.

flipdon
27th January 2003, 18:28
i'd like to bring my http://forum.doom9.org/showthread.php?s=&threadid=44202 (recent post) over here since it seems it could be a similar problem.

I get a problem when i have Dup() before Convolution3D. VirtualDub tells me i have an Avisynth read error when i try to move the slider. I don't get the problem when Dup is put after C3D, but wouldn't dup after C3D be bad, since there is smoothing. Is there something im missing here? I read a thread on the exact opposite of my problem, where they had errors when dup was after a smoothing filter like C3D ( http://forum.doom9.org/showthread.php?s=&threadid=37996 ).

LoadPlugin("F:\Avisynth\MPEG2Dec3_094.dll")
LoadPlugin("F:\Avisynth\Decomb.dll")
LoadPlugin("F:\Avisynth\Convolution3DYV12.dll")
LoadPlugin("F:\Avisynth\Dup.dll")
mpeg2source("G:\Flame of Recca\Recca3.d2v")


telecide()
decimate(15)
FieldDeinterlace()

Dup()
Convolution3D(1, 12, 22, 8, 8, 2.8, 0)

LanczosResize(640,480)

and if there's any suggestions to make about my script, please feel free to tell me (ex. deinterlacing, C3D or Dup settings). I've read i need to experiment alot with dup to get good results, but im not really sure which settings i should be playing around with yet.

sh0dan
27th January 2003, 23:00
When using a debug build of CNR2, dup and AviSynth, it always crashes at:

for (y = 0; y < pre->GetHeight(); y++)
memcpy(pre->GetWritePtr()+pre->GetPitch()*y,
src->GetReadPtr()+src->GetPitch()*y,
pre->GetRowSize());

I replaced it with
env->BitBlt(pre->GetWritePtr(PLANAR_Y),pre->GetPitch(PLANAR_Y),
src->GetReadPtr(PLANAR_Y),src->GetPitch(PLANAR_Y),
src->GetRowSize(PLANAR_Y),src->GetHeight(PLANAR_Y));

But it still crashes. Marc doesn't make the "pre" frame writable each time he tries to write to it, thus a null pointer is returned when he requests a writepointer.

He also tries to deallocate the frame, when his filter it destroyed, which it shouldn't (it is already being automatically destroyed). Therefore some people will recieve access violations on unload.

Basicly I'd advice a rewrite of the filter - it may have a good algorithm, but it's badly implemented, and will cause a lot of random crashes and other issues. BTW, the source and binary doesn't match.

I'll also check out the C3D issue in the other thread.

Guest
28th January 2003, 01:03
@sh0dan

Now you're really confusing me!

The pre frame is always made writable right after getting it. See the code I posted above. Are you saying that has to be repeated every time a write pointer is fetched on pre?

Also, I have it crashing on the pre GetFrame() call. Is that not happening for you?

sh0dan, please, we need to get to the bottom of this. I'd hate to have to shelve 2.5 work and withdraw my filters. But I can't have stuff crashing all over the place. All indications are that the cache has a problem. I see other combinations of filters failing in a similar way. And the MakeWritable thing is certainly distressing.

sh0dan
28th January 2003, 11:10
Where you get the crash depends on what frame is the first requested.

I can only reproduce the crash, in cases where the frame isn't requested and made writable (in case the if is false). The filter requests a writepointer to pre later, even though it has not been made writable.
Moving the makewritable out of the if-loop partially solves the problem, but it is still a problem that the filter tries to store the frame across GetFrame calls, which isn't allowed (the frame gets recycled).

Guest
28th January 2003, 15:15
The if test has the clause || !pre so it must have been run once and the pre is stored in what should be the lifetime of the filter.

Anyway, as I have said I think 3 times now, it is failing on the on the very first GetFrame call in the if clause. It worries me that you don't seem to care to try to understand that because this behavior makes 2.5 unusable.

Am I misunderstanding things or are you trying to frustrate me? :)

EDIT: Now, thinking about what you said... We can't put a PVideoFrame in our filter class data and have it remain stable throughout? It gets recycled?! Why? There should be an enduring reference to it. Please explain this in detail as I will have to redesign Dup and possibly others. Thank you for your patience.

sh0dan
28th January 2003, 16:21
(Check PM)

Frames need to be recycled, because that's how AviSynth controls the memory usage. It has always done so, so there is nothing new there. AviSynth 3.0 could very well change that, but that's far in the future.

WarpEnterprises
22nd April 2003, 15:54
Would it make sense to

1) make a calculate-only version of Dup
2) with an output that can be used in the ConditionalFilters
3) and - most important - that can take TWO clips as source

My goal would be to detect blended fields in some circumstances.
E.g. if you have

a - ab - b

then Dup detects motion between (1) a->ab and (2) ab->b.
If I would now compare ab to a merge of a and b there should be much less motion than (1) and (2) which can only be the case if it's a blend.

Bidoche
22nd April 2003, 16:39
@sh0dan

Are you saying that even VideoFrame copies you keep for yourself and never return to GetFrame can be altered ? I always thought those were safe. :/


AviSynth 3.0 could very well change that, but that's far in the future. 3.0 is not that far, the core is almost complete.
We can have a first version without the parser and the COM interfaces in a few weeks.

iwod
28th May 2003, 19:31
Since i can not found another thread about Dup therefore i will carry on with this one.

It seems version 2.2 is already in beta. May i ask what change is there apart from low level optimize??

I like this filter very much as it is a great help for anime encoding. However i don't understand why MarcFD say it is useless... was it a joke?? ( If it is then i don't get it.:confused: )

Guest
28th May 2003, 19:44
Originally posted by iwod
Since i can not found another thread about Dup therefore i will carry on with this one.

It seems version 2.2 is already in beta. May i ask what change is there apart from low level optimize??

I like this filter very much as it is a great help for anime encoding. However i don't understand why MarcFD say it is useless... was it a joke?? ( If it is then i don't get it.:confused: ) Dup 2.2 beta 1 has been around for a long time. It added low-level optimizations and fixed a problem with frame blending (not all the frames had equal weighting).

toraneko
7th June 2003, 09:13
@neuron2

first, thanks for this great filter

I'm dealing with an anime which has LONG period of non changing frames (sometimes several MINUTES)
could you please remove the limitation of 20 consecutive frames ?

Guest
7th June 2003, 11:19
Sorry, no.

EDIT: Two minutes would require me to fetch 2*60*30 = 3600 frames from Avisynth at a time. It's simply not practical. It could be done with a two-pass algorithm, but how may real-world clips have minutes-long duplicate runs? And even if they do, it's much more reasonable just to use Avisynth's FreezeFrame(), isn't it?

iwod
8th June 2003, 11:42
will 2.2 get out of beta? And there any more features planned to be added to Dup? I know you are hardworking on Decomb, so there is no rush. ( Thx for your Decomb and it IS GREAT :D )

Guest
8th June 2003, 14:13
I am always amused by people taking that "beta" word so seriously. If I simply rereleased the same code without the beta, you'd be happier?

Yes, I suppose I should go back and remove the beta moniker from all my stable filters. But until I do, you can rest assured that Dup is of release quality.

I don't plan any new features. What did you have in mind?

iwod
15th June 2003, 12:11
Hi, i have a new feature idea that i don't know if it is possible to implement.

It is poosible for dup to drop a frame instead of copy a frame if they are the same?

sh0dan
15th June 2003, 13:48
Why? You'll only desync your movies (as they must have constant framerate). Use decimate from the decomb package, which removes one of every x frames, based on several ways of finding the closest match.

iwod
15th June 2003, 16:02
Originally posted by sh0dan
Why? You'll only desync your movies (as they must have constant framerate). Use decimate from the decomb package, which removes one of every x frames, based on several ways of finding the closest match.

Well......... on the New A/V forum, it has come up with using AVIsynth with RV9, which support variable framerate.
So i intend to try what will RV9 react to that.......

sh0dan
15th June 2003, 16:47
AviSynth doesn't support variable framerate.

iwod
15th June 2003, 17:53
sorry for my poor english....... ( why is my english so poor even i live in brit?? )

I meant RV9 support Varible frame rate..........

I might have to rethink all these idea......

scmccarthy
15th June 2003, 20:06
I think AviSynth would need to tell RV9 what frame rate to use and it does that only once at the beginning, because AviSynth does not support variable frame rates.

Stephen

scmccarthy
15th June 2003, 20:23
After more thought, I realized that the point of DUP is to make it easier for the codec to compress the video, but it is up to the codec to decide whether it can drop a frame. Avisynth frame serves the uncompressed frames to the codec. When you open an AviSynth script in VirtualDub, it reports that every frame is a key frame, and that is true until the video gets compressed. So if you compress your video using a variable frame rate, it probably will drop the duplicate frames. It depends on the codec, not AviSynth.

Stephen

MightyKnife
17th August 2005, 22:12
Hello,

ok, now i'm playing gravedigger a little bit, as i'm just playing with the DUP filter. Some small ideas from me, if someone is still interested to maintain this filter.

1.) On one hand i'm missing a little bit a possibility in the debug-mode (show=true) to find the 32x32-block that's responsible for the highest percentage value.

2.) Is it possible to include a "guiding clip" that replaces the main clip in searching for differences in frames? Ok, to explain this: By deinterlacing anime-material captured from TV, one point that misleads the filter is the noise in the near of sharp lines. On the other hand, by simply resizing the video to half the size in height/width, this noise is much less present.
So why not use something like
DUP (mainclip, guidingclip)
with guidingclip being an arbitrary clip with the same length as mainclip, e.g. a resized, noise-reduced version of it. DUP analyzes the guiding clip for when to drop frames and then drops these frames from the main-clip.

Guest
17th August 2005, 22:39
The source code is available. Go for it!

MightyKnife
18th August 2005, 11:27
I know...

If i'm able to do that, i would do it ^^
But writing filters is still a little bit above my skills...

Was only a small idea...

Guest
18th August 2005, 13:31
@MightyKnife

Thanks for the ideas. I'll note them for future reference if I ever continue any development on Dup.

And I forgot to say...Welcome to the forum!

JnZ
3rd October 2005, 00:00
Hi, guys.

I'm litlle newbie with avisynth programming. I know basic scripts,but now, I have serious problem and can't solve it.

I have video (29.97fps,progressive) with some repeating frames (maybe bad IVTC). Video is very choppy. When I use Decimate(), some duplicite frames stays, and some non duplicite frames are cut off.

Following frames are same. I need to remove each one of them, to obtain video with non-repeating frames.
Frames: 1=2,8=9,13=14,20=21,25=26,30=31,32=33,37=38,42=43,49=50.......

I think that would be possible use DUP, but this is only detector, but I don't know, how to join DUP with some cutting function.

Any ideas. :thanks:

foxyshadis
3rd October 2005, 00:56
No, this is definitely not a dup-solvable problem. You need something like FDecimate (http://neuron2.net/fdecimate/fdecimate.html), which is much smarter than pure decimate, and you need to figure out what the real fps is by counting dups and dividing by your current fps.

If this only happens on some parts of the video, or if the actual frame rate varies significantly over the course of the video, you're out of luck (audio will desync if you use different decimations) and you just have to do what you can. Older films, especially silent ones, are often like that.

If these are capture-dropped frames, it might look worse to just cut them out, and the only way to restore smoothness would be to motion-estimate the duplicate frame between the two real frames.

AI
3rd October 2005, 04:23
JnZ
for "1=2,8=9,13=14..."
i.e duplicate frame every 7 frames
SelectEvery(7,0,1,3,4,5,6) (or SelectEvery(7,0,2,3,4,5,6))

But if 0=1,7=8.... (because first frame is "0")
then SelectEvery(7,0,2,3,4,5,6) or (SelectEvery(7,1,2,3,4,5,6))

---------------------
Add Later

if dup frames is not regularly
try GetDups by Fizick

if GetDups misunderstand lengh of dups frames =1 possible work:

Interleave (Last,Last,Last).GetDups(mode=0)

foxyshadis
3rd October 2005, 08:08
JnZ
for "1=2,8=9,13=14..."
i.e duplicate frame every 7 frames
SelectEvery(7,0,1,3,4,5,6) (or SelectEvery(7,0,2,3,4,5,6))

But if 0=1,7=8.... (because first frame is "0")
then SelectEvery(7,0,2,3,4,5,6) or (SelectEvery(7,1,2,3,4,5,6))
I was going to suggest something similar, but I noticed that some of the duplicates were 5 frames away (even one 2!), which is why I suspect it's a weird framerate conversion, or a bad capture. Your script would work fine for the first few cycles and then just cause more problems afterward; it really needs adaptive decimation.

I'll look into getdups, that sounds interesting.

JnZ
3rd October 2005, 09:35
No, this is definitely not a dup-solvable problem. You need something like FDecimate (http://neuron2.net/fdecimate/fdecimate.html),...
I've tried FDecimate,but not working well. So my idea is clear that duplicate frames in whole film and after that sync it to 23.976fps. (Never mind,that velocity of the film little change. Audio sync is no problem. I change it.

There is only duplicate frames, no missed frames. I only remark,that this is maybe bad captured HD video. I have some more, but only this is weird.
Other videos have duplicate frames every 5 frames, so Decimate() can restore it.

EDIT: I found some missed frames. So i must try FDecimate.
EDIT2: Fizicks GetDups returns only one of duplicate frame. :mad:

AI
4th October 2005, 04:27
EDIT2: Fizicks GetDups returns only one of duplicate frame. :mad:

I dont undestant you.

do You want delete duplicate frames?

PS My english is bad too ;)

foxyshadis
4th October 2005, 07:11
To simply get rid of every duplicate regardless of sync, try DeDup (http://students.washington.edu/lorenm/src/avisynth/dedup/). It's similar to dup but drops them instead of copying them across. It will output a timecode file, but you can always just ignore that if you're going to manually resync everything.

FDecimate will remain periodic, so if you clip really is squirrelly this might be the best way to get it. My dream filter is to a merged fdecimate, tdecimate w/vfr, and dedup, but that's not likely to happen. :p (Although I might be able to mod dedup to read tdecimate's output, saving me a pass.)

JnZ
4th October 2005, 21:00
Thx all for help,

but I think that this is no solution. This vido is realy crap. Some frames are double and some is missing.

I try DeDup, works fine, removes duplicate frames, but video is still jerky. (missing frames).

Ahyone thx all.

Guest
5th October 2005, 00:07
This line is off-topic for this thread. Please start a new one. Thank you.

Fizick
3rd January 2007, 15:14
neuron2 and sh0dan,

It seems that latest Dup v2.2.1 has a bug: mode blend=true does not work properly (nothing is blended really).
Result is not differ from blend=false.
(it was reported by smash94 at Russian forum)

Simplest script:
Dup(blend=true)

Older Dup v2.20b1 binary works fine.

I try complile them from source codes.
Compiled v2.20b1 (and v2.21) with VC6 provided project settings does not work too.
I tried look to source code, but is rather complex, so I have no ideas what is wrong.

But I found workaround. It works fine if optimisation (/O2) disabled or if compiler switch /Ot used instead.
VC7 (toolkit) works fine too.

May be you used different SDK library.
Please look.

Guest
4th January 2007, 04:31
* Fixes the broken blend=true mode. Stupid Microsoft compiler couldn't handle the code! I had to use the Intel compiler.

* Fixes crashes upon close of the application that opened the Avisynth script. That happened only when a log file was not defined.

http://neuron2.net/dup/dupnew.html

Fizick
4th January 2007, 11:41
Thanks for update.
But the same very active user smash94 :) also discovered second problem.
v2.20b1 and new 2.2.2 (after fixing a bug) have a little strange feature or bug.
Two-frame duplicate sequence is not really blended,
and generally for N frame sequence, only N-1 frames are blended.
Very first (original) frame of such sequence is not used for blending.

My suggestion is to change curent algorithm. All similar frames must be blended.
Suggested code patch to v2.22:

in GetFrame:
line 490: int c_div=32768/(planesY+1); // add 1 for normalization
line 503: int c_div=32768/(planesUV+1); // add 1

in mmx_average_planes function:
line 1157: if (planes<0) return; // (was plane<=0) to not skip very first frame
line 1189: jns kernel_loop; // (was jnz kernel_loop) to not skip very first frame

Opinions?

Fizick
19th January 2007, 21:47
May be I used wrong language?
Suppose we have similar (duplicate) frames 10 and 11. Other frames (9, 12) are different.
It is typical condition for anime.
The current version 2.22 (and 2.20) of Dup works so way:
It detects that frames 10, 11 are similar,
but does NOT blend them (but put message about blending in show mode).

Guest
19th January 2007, 23:54
The problem is that sh0dan low-level optimized my original C code and I don't have the knowledge of what he did (or the time to analyze it) to determine if your changes are valid. Can you PM him about it?

Fizick
20th January 2007, 02:16
Thanks for your answer, now I understand (will try PM him).
Do you have original C version (probably v2.0 or 2.0.1) to compare?
The question is: did you have an intention to skip (do not blend) first frame in sequence (which may be lesser quality)?
For my noisy anime it is not a case trough.

G_M_C
25th January 2007, 19:36
Hi guys, im currently working on a clip that's 59,940 fps and has a irregular pattern of duplicates. I've put Dup (show=true) in my script, and find that it correctly finds every duplicate throughout the movie (except on some very slow fades; I'll try to exclude chroma on those parts, see if it helps ).

But my question is; Since dup finds the duplicates, allmost without errors, is it somehow usable to remove the duplicates. I hope to replace a string of duplicates with exactly one blended one (effectively removing the dupes).

When i've done my math correctly i should be left with a correct 24 fps clip (mesured the old fashion way with pen and paper noting AAABCCDDEEEFGHHIJJ etc etc. No regular pattern for several complete alphabets :P)

foxyshadis
25th January 2007, 20:10
You can try DeDup, and just throw the timecodes away. You're almost guaranteed to throw off sync though, because it won't be perfectly accurate. You should probably use TDecimate with a high cycle and cycleR to compensate for the irregularity. (Or FDecimate, which is similar.) That could also lead to sync drift if you have long runs of dups, though.

Guest
25th January 2007, 20:16
FDecimate() won't cause sync drift.

G_M_C
25th January 2007, 20:51
You can try DeDup, and just throw the timecodes away. You're almost guaranteed to throw off sync though, because it won't be perfectly accurate. You should probably use TDecimate with a high cycle and cycleR to compensate for the irregularity. (Or FDecimate, which is similar.) That could also lead to sync drift if you have long runs of dups, though.

I've used both T and F Decimate with several settings. The best result i had was 1 dupe every 20 frames varying between 17 to 25 or so. I havent got a clue what's up with the source, been strugling with it for a long time now.

And speaking of sync audio; I'm at the point that I will try to sync the audio seperately through Audition or Nuendo, combined with a AC3 encoder (working roughly chapter by chapter and checking + correcting).

So i'll try DeDup first, and see if that option is working (finally).

But before i forget: Thx guys for your help and tips. Btw. the Chroma-false option seems to work very well on the slow fade :) Great dupe-detecting-plugin Neuron2 !

foxyshadis
25th January 2007, 21:41
FDecimate() won't cause sync drift.

I meant within a single cycle, if it's long enough, sorry.

Guest
25th January 2007, 21:56
Yes, I knew you meant that. :) I just wanted to make sure others knew that FDecimate() isn't cycle based.

G_M_C
26th January 2007, 09:40
DeDup seems to work out fine, better than i thought;

I finetuned the settings and from 59,94 fps i came out on 23,8fps .... allmost right. Never would have guessed that if would work out this good.

I'll retry FDecimate () tonight, maybe i did something wrong before and/or misunderstood the FAQ (reading so many FAQs in English isnt allways easy for me).

Fizick
6th March 2007, 21:20
I decide to release Dup version 2.23 with above mentioned bugfix:
first frame in the string of duplicates is included in blending too (now).
http://avisynth.org.ru/dup/dup223.zip

Please test

Guest
6th March 2007, 22:24
I decide to release Dup version 2.23 Did you ever hear from sh0dan about it?

Fizick
6th March 2007, 22:54
I sent PM to him 20 january, without answer.
Seems, he is busy with great SoundOut plugin :)

IMO, the question is: to use or not to use first frame in string for blending. I vote for "to use" and ask users to test.

plugh
13th February 2008, 17:34
a blksize argument was added to Dup, which allows 8, 16, or 32.

I'm requesting also 64.

Rationale: In my experience using tdecimate (which supports blockX and blockY args for metric blocks) with HD material, I find that 64x64 works much better than the default of 32x32. Thinking about it, this seems to make sense when you consider how much of a frame is subtended by the metric block.

Take your typical 16x9 SD frame - 640x352 - and consider the area covered by a 32x32 block. Now consider an HD frame of 1280x704, and that same 32x32 block covers one fourth less of the frame area. To get the metric block to cover the same relative section of the picture, it needs to include four times as many pixels - ie 64x64.

As is pointed out in the docs "The lesser block size, the greater sensitivity to small details change (and to noise)." My take on it is that this should be considered relative to the frame size - a 32x32 block on HD frame is analogous to using a 16x16 blocksize on SD frame.

Thanks for your consideration.

Guest
13th February 2008, 18:35
I have a lot of work on my plate right now. The source code is available. Can you please add it and send me the changes? Thanks.

Kumo
13th February 2008, 19:06
does this filter work by itself(calling it in the script),or do i need to configure the encoding settings in a certain way(variable frame rate or something like that)?

Adub
13th February 2008, 19:34
Call it in the avisynth script.

plugh
13th February 2008, 19:38
Pulled it down and took a quick look. The blocksize stuff seems to be tied to the mmx / isse code.

That would be a big learning curve for me in order to do the mod.

Am I correct that Fizick did the extension to the smaller blocksizes?

Guest
13th February 2008, 22:28
Am I correct that Fizick did the extension to the smaller blocksizes? That is correct.

Kumo
14th February 2008, 18:41
i can't understand how this filter work.if it finds 5(i.e.) equal frames(at least very very similar),it discard the firts 4 to keep just the last one.does it mean that the fps in that passage will be less than the original(i.e. less than 23,976),1 frame will last for the time 5 were suppose to last?that way,will the encoded file be variable frame rate?what kind of default/safe settings can i specify in my script to give it a try?

foxyshadis
14th February 2008, 19:08
DeDup works that way. Dup just copies the frames to make them 100% dupes.

Kumo
15th February 2008, 11:12
DeDup works that way. Dup just copies the frames to make them 100% dupes.

what's the point in making dupes?how can be the bitrate lower in that way?

Adub
15th February 2008, 12:09
The bitrate doesn't lower. It's just the same amount of bitrate.
With exactly duplicate frames, the difference between each frame is (obviously) null. So the compensation usually involved with the transition from one frame to another is not necessary.

Normally, when a new frame is drawn, it just takes all of the stuff that hasn't changed from the previous frame and adds the stuff that needs to be changed, this resulting in less info needed to store 2 frames. Think about it, you draw the first frame, then you only need to draw the stuff that is different in frame 2, and use references to the first frame for the rest.

What Dup does is make frame 2 and exact copy of frame 1, thus resulting in no change, meaning essentially storing 2 frames for the price of one.

Note: I am not 100% sure on this, but I think it's right.
Edit: Okay, I was slightly mistaken. With the averaged frames used to create a new dup frame, noise is essentially removed. No noise = lower bitrate. That is the reason for the bitrate reduction.

Kumo
15th February 2008, 13:20
now i see, thanks.should i put "dup()" at the beginning or at the end of the script?does it work better on untouched or denoised frames?

Adub
15th February 2008, 22:08
I think it would depend on both your source and your script. Post your script and either describe your sources, or post a small sample.

Kumo
16th February 2008, 10:22
the source is a japanese ntsc r2 anime dvd.here is my script
DGDecode_mpeg2source("E:\kor\VTS_01sample.d2v",info=3)
ColorMatrix(hints=true)
o = last
ox = o.width()
oy = o.height()

crop_L = 4
area_L = 32

crop_T = 4
area_T = 32

crop_R = 4
area_R = 32

crop_B = 0
area_B = 0

stackvertical(
\ stackhorizontal(
\ o.spline16resize(area_L,area_T,crop_L, crop_T,area_L-crop_L,area_T-crop_T),
\ o.spline16resize(ox-area_L-area_R,area_T, area_L,crop_T,ox-area_L-area_R,area_T-crop_T),
\ o.spline16resize(area_R,area_T,ox-area_R,crop_T,area_R-crop_R,area_T-crop_T) ),
\ stackhorizontal(
\ o.spline16resize(area_L,oy-area_T-area_B, crop_L,area_T,area_L-crop_L,oy-area_T-area_B),
\ o.crop(area_L,area_T,-area_R,-area_B),
\ o.spline16resize(area_R,oy-area_T-area_B, ox-area_R,area_T,area_R-crop_R,oy-area_T-area_B) ) )

source = last
backward_vec3 = source.MVAnalyse(isb = true, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec3 = source.MVAnalyse(isb = false, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400,idx=1)
dfttest(sigma= 0.8)

LimitedSharpenFaster(ss_x=1.1, ss_y=1.1, smode=4, strength=150,undershoot= 10)
dup()
are others filters in the correct order?thanks

foxyshadis
16th February 2008, 23:03
That's a good place to put dup. You could put it before sharpening as well, as long as it's after deinterlacing and denoising.

Merlin, it's actually for both of the reasons you listed. ;)

Adub
17th February 2008, 00:18
I understood that, it's just that Kumo wanted to know about the reason for the lower bitrate, and bitrate specifically had to do with the noise, while overall size dealt with the duplicate frames.

And thanks.

Kumo
17th February 2008, 11:55
That's a good place to put dup. You could put it before sharpening as well, as long as it's after deinterlacing and denoising.

Merlin, it's actually for both of the reasons you listed. ;)

will it speedup the process if i put right after denoisers?i mean, will others filters be faster working on dups instead of original frames?

foxyshadis
17th February 2008, 18:02
Since the sharpeners and other random things at the end (in your other thread) are purely spatial, it won't affect speed at all.

Kumo
19th February 2008, 21:48
i'm trying to use dup in a script to encode an '80s japanese anime dvd.i get jerky video ,specially in slow zoom or vertical scrolls(calling it with default settings).how could be an ultra-safe(to avoid jerkyness) setting?would it help in lowering wasted bitrate anyway?

Adub
19th February 2008, 23:50
Your encoding to x264, correct? The fact is that x264 handles duplicate frames so well that it almost isn't worth it to use dup. The only real benefit is when it comes to denoising.

egrimisu
2nd July 2008, 22:56
Where to put DUP in this script + if i put something in wrong order tell me:

DGDecode_mpeg2source("D:\Work Hanbun no Tsuki ga Noboru Sora\Hanbun_no-Tsuki_ga_Noboru_Sora_01.DVD(MPEG2.WAV)[Misu]_Track1.d2v", cpu=0)
setmtmode(2)
dup()
setmemorymax(1024)
SmartFade(dgm=true)
source = last
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=1)
fluxsmoothST(5,5)
fft3dfilter(sigma=0.8, bt=1, bw=32, bh=32, ow=16, oh=16, plane=4, dehalo=0, ncpu=2)
ttempsmooth()
#FastLineDarkenMOD(strength=16,thinning=0)
#limitedsharpenfaster(Smode=4)
#maa()
gradfun2db(thr=1.2)
crop (4,0,-4,0)
Spline36resize(720,480)

SilentTweak
12th December 2008, 04:36
Is Dup considered fairly dangerous to use? Like I want to use Dup(Threshold=2.1), does anyone see any problems this might cause with Anime? Like maybe rape mouths? Or is that unlikely to happen? Like for example I want to use Dup to fix frames like these: http://rapidshare.com/files/172566014/DupFix.rar.html 01 is the broken frame 02 is the next frame being dupped for 01. So 01 becomes 02 when using Dup. Thing I'm worried about is if Dup causes any problems with miscalculating mouth movement at a setting of 2.0 to 3.0.

thetoof
12th December 2008, 17:30
High threshold is more likely to introduce losses in motion fluidity (jerks (especially slow pans and zooms), small movements will disappear (mouth movement...), etc...)
There was an attempt at creating a motion adaptative dup function some time ago:
# Dupped() by Corran
#
# This filter requires a YUV source. Use converttoyv12 if needed
# before calling dupped()
#
# thresh = Used to determine when to delare a frame as new
# Use stats=true to help determine the best value for your source
# Lower number = more sensitive. (Default=16)
# panthresh=1.7 This is used to determine when to consider a frame
# part of a low motion scene. (Default=1.65)
# stats = Enable/disable display of stats. (Default=false | Not shown)
# (Last = last frame, this = this frame, next = next frame)
# showme = Enable/disable display of video that most stats are
# derived from. (Default=false | Not shown)
#
# Do not use with SetMTMode(). This function requires the frames to be
# accessed sequentially. Instead, use MT() to mulit-thread on a per-filter basis.
#
# Example:
# dupped(thresh=20,panthresh=1.7,stats=true,showme=true)

function dupped(clip clip, int "thresh", float "panthresh", bool "stats", bool "showme")
{
global a = default(clip, last)
global thresh = default(thresh, 16)
global panthresh = default(panthresh, 1.65)
global stats = default(stats, False)
showme = default(showme, False)

function replaceframe(clip clip, int oldframe, int newframe)
{
a = default(clip, last)
trim(a,0,oldframe-1)++Trim(a,newframe,newframe)++trim(a,oldframe+1,Framecount(a))
}

b = a #for later use with showme conditional (it isn't passed to scriptclip)
global newframe = a
a = scriptclip(a,"""
b = a.duplicateframe(1)
d = a.deleteframe(1)
c = subtract(a,b)
YMinMax = YPlaneMinMaxDifference(c)
YAbvMed = YPlaneMax(c)-YPlaneMedian(c)
YBelMed = YPlaneMedian(c)-YPlaneMin(c)
YAbvBelMedDiff = Abs(YAbvMed - YBelMed)
new = (YMinMax>=thresh || YAbvBelMedDiff>=10)
YMinMax2 = !new ? YPlaneMinMaxDifference(subtract(a,d)) : 255
YMinMax3 = (YMinMax2<=thresh ) ? YPlaneMinMaxDifference(subtract(b,d)) : 255
pan = (YMinMax3 != 255 && YMinMax3!=0 && YMinMax3>=(YMinMax+YMinMax2)/2*panthresh)
stats ? subtitle(a,"YMinMax(last,this) = "+string(YMinMax)).\
subtitle("YAbvBelMedDiff(last,this) = "+string(YAbvBelMedDiff),y=54) \
: a
newframe = (current_frame<=1 || current_frame>=Framecount()-1) ? current_frame : newframe
stats ? !new ? subtitle("YMinMax(this,next) = "+string(YMinMax2),y=18) : subtitle("YMinMax(this,next) = Null",y=18) : last
stats ? (YMinMax3!=255) ? subtitle("YMinMax(last,next) = "+string(YMinMax3)+"|"+string(ceil((YMinMax+YMinMax2)/2*panthresh)),y=36) : subtitle("YMinMax(last,next) = Null",y=36) : last

newframe = (new || pan) ? current_frame : newframe

stats ? subtitle("Last New Frame = "+string(newframe),y=90) : last
(stats && pan) ? subtitle("(Slow pan/zoom or action detected)",y=108) : last
(current_frame != newFrame) ? replaceframe(current_frame,newFrame) : last

return last
""")
a = showme ? stackvertical(a,subtract(b,b.duplicateframe(1))) \
: a
return a
}
Have fun testing! Dupped(settings)

wyti
12th December 2008, 19:01
or if you want to not taking risk, use a low blksize (8 if possible)
so small movements are detected pretty well (a mouth move does about 4% IIRC)
so you don't have any problem with the treshold = 3.0 and don't kill small movements