New duplicate detection filter

[Guest]

@cult

We forgive your ignorance. :) :)

Belgabor and Dark-Cracker said it all.

[Guest]

Here is Dup() version 1.2, which properly fixes the Avisynth stack overflow problem.

@Suikun

When I complete the release of Decomb 4.0, I will consider your suggestion.

[Dark-Cracker]

hi,

just a little question, to obtain the best result where in the avs script should i use dup ? at the end of the script (after the crop resize,denoise,smooth,sharp..etc....) or just after the resize ?

thank u, bye.

[Defiler]

It seems to me that you'd want to use dup() as early as possible.. right after the deinterlacing? That way temporal filters that you run later in the script won't be making changes based on data that clearly wasn't worth keeping. Also, if you run dup() after smoothing, adjacent frames that are similar, but not identical, will be even more similar. (Because smoothing discards data, they will have more bits in common.)
Am I wrong?

[vidiot]

Fast changes aroud here...
I could have put my tests into this thread (must have been blind that day...)

@neuron2:
You leave the noise parameter out since V1.1,
i know you suggested not to change it anyway, but...(why)?

@MarcFD:
Even with lot of homework to do - just stay here and improve your filter plugins. ;)
I know that for example CNR2() was not you idea from the ground,
but with your input it really rocks (speed depending)!

Why not putting some more effort in copysame?
Your ideas are great! And that is what counts.
With a little more debugging and the better "readme´s"
you recently startet with - your plugins were killers.
Ask neuron2 how much time he spent with decomb!
This plugin alone kept me capturing from tv - otherwise I would have given up, because of the time it needs to process.

Thankyou for all those plugins!

Harald

[Guest]

@vidiot

With the revision to the algorithm the nr parameter is no longer required. My theoretical considerations as well as my testing confirms it.

[stickboy]

Originally posted by Defiler
It seems to me that you'd want to use dup() as early as possible.. right after the deinterlacing? That way temporal filters that you run later in the script won't be making changes based on data that clearly wasn't worth keeping.If you run a temporal smoother after dup, wouldn't that introduce differences to duplicate frames and prevent them from being perfectly identical?

[ronnylov]

I have been playing around with Layer to average two captures to decrease noise and now I wonder if this method could be used on "identical" frames? What I mean is not only duplicate the frames but before duplicate them make an average of all frames that are "identical" (the noise is not identical and averaging them should decrease the noise).

Maybe this is done with temporalsoften with low thresholds already?

[Defiler]

Originally posted by stickboy
If you run a temporal smoother after dup, wouldn't that introduce differences to duplicate frames and prevent them from being perfectly identical? Not unless the temporal smoother is broken, because there should be exactly zero temporal changes between those frames.

[stickboy]

Originally posted by Defiler
Not unless the temporal smoother is broken, because there should be exactly zero temporal changes between those frames. But couldn't they be smoothed with the frames around them? Let's say you have a sequence of frames A B1 B2 C, where B1 and B2 are identical. After temporal smoothing, couldn't A affect B1 and C affect B2?

[Defiler]

OK.. Run it at the beginning and end of your script, then. :)

[ronnylov]

Originally posted by stickboy
But couldn't they be smoothed with the frames around them? Let's say you have a sequence of frames A B1 B2 C, where B1 and B2 are identical. After temporal smoothing, couldn't A affect B1 and C affect B2?

Yes, and what I meant was that I was looking for a smoother that averages the pixels of frames B1 and B2 only (which are identical except for the noise) and then make new duplicates of the averaged frames.

[stickboy]

Originally posted by ronnylov
Yes, and what I meant was that I was looking for a smoother that averages the pixels of frames B1 and B2 only (which are identical except for the noise) and then make new duplicates of the averaged frames. I'd like to see this behavior too. (I'm still waiting for a similar feature to Decimate...)

[Dark-Cracker]

hi,

to sum up it was better to use dup after the temporal smoother and after the temporal soften, just put the dup filter at the end :) i am right ? (sorry to ask u again this question but my english was very poor and i don't have understand fully the 3 previous post :) )

thank u.

Bye

[meleth]

I was thinking, wouldn't it be great if dup() had it's own noise cleaner? Or if this strategy was implemented into a noise filter. It just strikes me that it would be more time efficient if the filter cleans the frame before copying it, instead of another filter cleaning it twice.

Also, i got a question. Where in the filter chain would be the best place for dup? Before or after noisecleaning, cropping, ivtc, resize, sharpening etc.?

I'm currently using this order: Ivtc, dup, crop, sharpen, noise, resize, anti-rainbow.

[Guest]

@meleth

I am currently preparing a new Decomb release. I will actively return to Dup() and this thread when it is done.

[meleth]

alright cool, i'll stay tuned then :p

[Belgabor]

You shouldnt use any frame duplicator (be it Dup or CopySame) before temporal noise reduction. You siply loose the benefit of the noise reduction on copied frames, which is in fact where it could have its best benefit.

Cheers
Belgabor

[Neoze]

I use the dup() filter in an other way:
to calibrate the filter and their parameter:

The idea is to use dup() beetween to frames: the original and the filtered.
s=ImageSequence("Image%d.jpg",0,7,25).converttoyuy2()
d=s.filter (parameter)
i1=trim(s,1,1)
i2=trim(s,2,2)
../..
j1=trim(d,1,1)
j2=trim(d,2,2)
../..

video=i1+j1+i2+j2 ...
dup(video,threshold=5,show=true,debug=true)
(full post here: http://forum.doom9.org/showthread.php?s=&postid=199123)

That way dup() give you a statistic beetwen the 2 picture and focus on the main difference.
But in this usage It will be more convenient if dup() display more than one area where it find the bigger difference.
That is my request: displaying a parametic number of areas with a color scale.

[trbarry]

I have been playing around with Layer to average two captures to decrease noise and now I wonder if this method could be used on "identical" frames? What I mean is not only duplicate the frames but before duplicate them make an average of all frames that are "identical" (the noise is not identical and averaging them should decrease the noise).

I have been scrolling through 4 pages of this thread (I haven't been following) to see if someone would propose that. It seems like it would be a usefull idea, but more work.

- Tom

[WarpEnterprises]

Has somebody else encountered problems with DUP and dark parts / fadings?
There it seems to copy frames too readily.
But maybe this is a principal problem because in a fade the pictures are the same except their luminance.

Nevertheless: How to handle this?

And of course I'm pleading for a "blend" instead of "copy", too.

[Marc FD]

IMO, bframes achives almost the same result than any dup or copysame in terms of compressibility. That's why i've stopped devellopement on CopySame. And trust me, i've tested and tweaked both a lot, they failed in the same scenes and when i saw the results, i've immediatly removed CopySame of my site. I think CopySame is a bad thing. I feel like a crazy scientist when i think i've invented it. it's way too destructive.

Dup was created to improve Decimate of Decomb. and Decimate is very accurate now. i think it's the only good use of this type of filter.

i know some users don't care at all or don't trust me, but i want to say the users of Copysame, that i strongly recommend NOT to use this filter. because it could create problems they won't notice first, but would be very unpleasant to eye when they would watch it later.

the same goes with dup. but i've too much respect for Donald Graft to ask not to use his filter.

I'm really sorry for any image degradation Copysame could have created, and i feel the need to apologise for this.
MarcFD

[Xenoproctologist]

Originally posted by Marc FD
i know some users don't care at all or don't trust me, but i want to say the users of Copysame, that i strongly recommend NOT to use this filter. because it could create problems they won't notice first, but would be very unpleasant to eye when they would watch it later.

the same goes with dup. but i've too much respect for Donald Graft to ask not to use his filter.

I'm really sorry for any image degradation Copysame could have created, and i feel the need to apologise for this.
MarcFD
The trick to not getting image degredation (e.g. sudden "pops" when the image exceeds the set threshold; or decimation of slow pans/zooms) is to use extremely low threshold values. I've found 0.5% to be the threshold just below the point where the "pops" become noticable in low-noise source. Above 0.8%, the aforementioned pan/zoom decimation starts to occur.

Dup is useful--you just can't wield it like a sledgehammer.

[Marc FD]

the gain in compressibility with 0.5% would be so ridiculous, that i don't deserve the additionnal processing time and the risk (even very low threshold can copy slow fades and make ugly results)

if you really want to gain compressibility, use the Temporal Cleaner that vlad is going to port to avisynth. i think it's exactly what you need. a pixel locker would improve more compressibility, but would be risk free.

when i've created CopySame, i wanted to create N_vops with XviD. with some tweaking in XviD, i could do it, and have a real boost in compressibility. but without that, the gain is really poor.

[Defiler]

I always assumed that CopySame was just a testbed to perfect the process by which frames would be marked as "zero frames" in XviD. Was I wrong?
It still seems to me that zero-frames in XviD will be very useful.
Will this functionality be included in XviD directly, or will we still need things like CopySame?

[Marc FD]

>I always assumed that CopySame was just a testbed to perfect the process >by which frames would be marked as "zero frames" in XviD. Was I wrong?

in fact copysame was a proof of concept.

>It still seems to me that zero-frames in XviD will be very useful.
>Will this functionality be included in XviD directly, or will we still >need things like CopySame?

I've tested the whole thing a lot before saying it's crap. A bframe, even with the same quant, would be much better than a zero-frame. it would be a bit bigger, but would be 100% accurate. in fact zero-frames have no interest as soon as you use bframes. it's why the gain of Copysame with DivX 5.02 pro was so ridiculous. because DivX was already using bframes on potential zero-frames.
if XviD with bframes encounters a real or potential zero-frame, it would create a incredibly small bframe. on my anime encodes, i saw bframes who are 1 byte big ! a zero-frame would never be better.

in short, zero-frames are history. bframes rules. syskin's dynamic bframes detection rules. XviD rules.

[Xenoproctologist]

Originally posted by Marc FD
the gain in compressibility with 0.5% would be so ridiculous, that i don't deserve the additionnal processing time and the risk (even very low threshold can copy slow fades and make ugly results)

if you really want to gain compressibility, use the Temporal Cleaner that vlad is going to port to avisynth. i think it's exactly what you need. a pixel locker would improve more compressibility, but would be risk free.

I care less about compressability than visual quality. If Dup is taken to the next logical step in its evolution--averaging sequences of duplicate frames--it could do wonders for cleaning low-budget anime with high frame-repeat counts.

And as to Temporal Cleaner, I stopped using it a long time ago because it had a tendency to cause really nasty temporal artifacts--most noticably during slow fades.

[Zarxrax]

You were likely using too high values in it then. If you use values of 5 for blending threshold and 3 for pixel lock, I doubt you would ever see any artifacts.

[Guest]

I'd like to point out that I have achieved 40% savings on some clips using Dup().

When Suikun's idea above of copying on a per-block basis is implemented, I think nobody will doubt the utility of the idea. Blending, too, will increase the utility of the filter.

[WarpEnterprises]

@neuron2: sorry for being repetitive but what about my problem with fades?

[ErMaC]

I think in order to really get the most out of Dup you have to be willing to tweak it on a scene for scene basis like so:

function yes(clip)
{
return clip.Dup(copy=true,threshold=2)
}
function no(clip)
{
return clip.Dup(copy=false,threshold=4)
}
s=AVISource(blah)
v1=s.Trim(0,116).yes()
v2=s.Trim(117,433).no()
v3=s.Trim(434,554).yes()
...etc,,,


What I would do is encode the entire movie with Dup at the desired threshold I want, and whenever problems arise in a section, I tweak that specific segment in my AVS file. It's feasible when you're only encoding <6 min long clips, but definately would be way too much work when encoding, say, an entire 25 minutes episode.

I think what Dup could really benefit from is a more advanced detection engine which would not classify dupes on pans and fades. If safeguards could be implemented to catch these two cases, then I think that would make the filter far more easy to use as those are the two cases where the problem arises the most.

Just thinking off the top of my head (I am not an expert here, so it's entirely possible this won't work), you could implement a comparison between potential "copy" frames to check for pans or fades. For pans, check to see if there's an almost uniform % change amongst all the blocks analyzed or maybe use a motion detection algorithm to see if every block's image is moving at the same vector. For fade detection, just check to see if there's a uniform change in Luma across every single block. Crossfades would be more difficult to detect but I think because you're not fading to black the chances of detecting a duplicate there are lesser.

Quick question: Does Dup analyze the chroma channels at all, or does it analyze the Y channel only? Could a chroma=true argument be implemented like in Telecide that would give better, but possibly slower, results on certain sources?

Edit: Oh and I may have found a possible bug... When I set MAXCOPIES to 1, I don't think the behavior is what it's supposed to be. What I would expect is that the maximum # of frame copies to be created is one, so for instance on 12fps footage from a 24fps stream I would ensure that I never accidentally copy a new frame. However, setting MAXCOPIES to 1 does not seem to achieve this behavior and instead seems to copy EVERY frame (or so the show=true says) from the previous one and basically phase shifts the entire video stream one frame forward. Is this the designed behavior? Am I understanding the information it's giving me incorrectly?

[High Speed Dubb]

In the case of fades, I think block-based motion detection could be something of a liability. The blocks are giving greater power to detect motion which occurs in a small part of the screen. But when the motion is slow and occurs throughout the screen, using only the block with the greatest change is ignoring much of the available information.

[JuanC]

I use "dup" a lot when encoding animes. It's a great tool. I normally "fine tune" the threshold by selecting problematic scenes where some of the not-to-be-copied frames are being copied. So typically I end up using real numbers (not integer). That's when I get problems:

If I use a float for threshold combined with the "show" option, With some of my AVS scripts VirtualDub will eventually stop displaying the video, and will show a message in the status bar like this: "AviSynth read error: AviSynth: caught an access violation at 0x01eb0894, attempting to write to 0x82128033". The good part is that there are no problems when I finally use any float for threshold but I use the "copy" option instead of the "Show" one.

The workaround: I noticed this NEVER happens with the show option when I use the closest integer!! to my preferred float threshold.

Thanks Donald :) :J

FYI: This has happened with AVIsynth 2.06 and more recent cvs snapshots (12-nov). I am encoding to DivX502 some animes I captured from live tv broadcasts. Before dup I use decomb (ivtc), resize to 368x208 (16:9) and use _2dCleanYUY2 to filter some noise.

May be the size (h&v) must be module 32?? Here's one of my scripts

VIDEO=mpeg2source("D:\Paulita\HB24.d2v")
audio=WAVSource("D:\Paulita\HB24.wav")
AudioDub(video, audio)

Telecide(guide=1,gthresh=10,blend=false)
Decimate(mode=0,cycle=5,quality=3)

crop(0,60,640,360)
BiLINEARResize(368,208)
_2DCleanYUY2(0,6,2,2,0,4,4)
Dup(threshold=2,show=true)

[ErMaC]

Actually I reported this bug along with a slight behavioral modification suggested to Don a few days ago - he's got a fix worked out that should be released soon (v1.4 I believe he told me) but he asked me not to give out the DLL until his official release.

Apparently what was causing that problem was that if show=true, and your resolutions aren't mod32, if the "box" that has the largest change is outside of the frame it wasn't catching that and so essentially drawing the box outside of the video which was throwing the exception. He just added some bounds checking so it doesn't draw outside the frame anymore.

[Guest]

ErMaC is quite right. Thanks to both of you for bringing this to my attention. I have attached the fixed version Dup 1.4.

[ErMaC]

Wow less than 10 minutes after I make my post :D

Thanks again Don, and I hope once you get the new Decomb YV12 stuff worked out you can return to working on this filter cause I think it's really nifty.

BTW for everyone else, since it seems like it's not pointed out in the readme that comes with Dup there, the other change in the release is the behavior after MAXCOPIES is reached.
Before when the MAXCOPIES # of dups was reached, the next frame was compared against the previous original (so in a string of dups with MAXCOPIES=4, frame 6 would be compared against the original frame 5 and would then copy it). The only difference now is that instead of being compared, frame 6 is just passed through as is. This most of the time won't make too much of a difference, except when you use very low MAXCOPIES numbers like 1 or 2.