Automatic replacement of bad frames in scene changes? need help.

[Stigma]

Lately i have aquired several anime DVD's wich have white dirty lines at the top and the bottom of the picture whenever the scene changes. This is on the original material, so theres not much to do about it exept replace the frames with good ones. (you anime encoders out there probably know what im talking about. Berserk has very large (and thus annoying) scene change lines. I think its somethign with how the anime is shot, but thats not the point here..

The this is that i CAN edit this and make it look very good by a simple script. The frame right before and right after a scene change is allways bad, so if i have scene changes at frame 10 and 80, this script will fix it:
a=10
b=80
Freezeframe(a,a,(a-1))
FreezeFrame((a+1),(a+1),(a+2))
Freezeframe(b,b,(b-1))
FreezeFrame((b+1),(b+1),(b+2))

Problem is that it takes a LOT of time to manually find these scenes, and doing this on 26 episodes, not to mention the hundreds of DBZ eps, is incredibly slow and tedious.

so.. is there a way i can somehow link this to some scene change thing like the DivX codec uses to find out where to place the keyframes? Because the errors only appear in scene changes i could automate this and get a perfect result.

Or.. if there is another used method to deal with this that i dont allready know (not nessecarily including AVS scripting) i would like to know.

Im very computer littarate in general, and im an experienced encoder, but i have very little coding experience, so any solution would have to be within the frames of other programs i guess as i am currently unable to build my own.

Grateful for all contructive response =)
-Stigma

PS: i have attached an image of the white stuff for those that are curious about what the heck im talking about ^_^

[Dreassica]

Hmm, i usually do that manually in Aviutil adn then feed it to avisynth using VFapi. Dont know if it can be done with avisynth though. A filteer that specifically tries to smooth and/or remove blending artifacts, that are prevalent in PAL animes too would be nice too.
Problem is that that will be nigh on impossible to do.

[wotef]

the dupe could help you identify the scene changes frame numbers, which you could then feed into your trim script

[Stigma]

I have never used aviutil, are you saying that it has a way of detecting scene changes? If thats the case then i could just feed (manually atleast) the rame numbers into my script..

-Stigma

[SansGrip]

It wouldn't be difficult at all to write a filter with a scene change detector that simply copies over frame new_scene with frame (new_scene + 1).

Maybe someone with a shorter to-do list than myself could take a look at it... ;)

[Dreassica]

Originally posted by Stigma
I have never used aviutil, are you saying that it has a way of detecting scene changes? If thats the case then i could just feed (manually atleast) the rame numbers into my script..

-Stigma

It does have it yeah, but i do it manually. Like to have control over it myself, so i never really used it.

[Stigma]

well of course, it would be near impossible to get a 100% perfect filter for this, since the errors vary in size, how they appear ect.
BUT.. it wouldbe a deinate boon to those that dont bother to frameedit in the first place... it would atleast get more then half i bet

So is there a coder out there somewhere who wants work? ^_^

-Stigma

[SansGrip]

well of course, it would be near impossible to get a 100% perfect filter for this, since the errors vary in size, how they appear ect.

It would be very difficult to remove the errors from the bad frame, yes, but it would be very easy to completely replace the frames immediately before and immediately after a scene change.

In other words, given these frames:

A B C D - E F G H

where the dash represents the scene change, we assume D and E are bad. D gets replaced with C and E gets replaced with F.

[Stigma]

Well it would certainly be a HUGE timesaver, but is anyone out tehre willing to make this thing? I know i cant yet =/ i dont have the slightest idea how to use a scene detection thing in combination with this.

-Stigma

[sk8r4life]

Yea this would be very helpful and timesaving in a sense.... but the last 2 frames in every scene change are always the same, ie any motion or scrolling scene. For a filter to be effective in doin this task, it would have to work in compilation with AviUtil's scene change detection on very low settings.... which takes forever to go throu a whole episode

In short, a filter like this would be a big advance in quality of animes being ripped/encoded, but very unlikely..... very impossible if u ask me. Take the time to frame correct it yourself, if u only want to hit scene changes, use AviUtil, very useful and hits about every scene change, (plus any big change in the frames) or go throu the whole episode by hand and fix the errors...

[Tanma]

Well, it has been almost four years, does anyone have a solution or can create a solution for this kind of problem exposed by Stigma?

I tried to use freezeframe manually, just for test, and it took 4 hours for a 22 minute anime! I had to freeze one frame before and after each keyframe.

For example, I have a bad keyframe = 20.
freezeframe(20,20,19)
freezeframe(21,21,22)

In other words:
freezeframe(a,a,a-1)
freezeframe(a+1,a+1,a+2)

If some plugin could catch ALL the keyframes when opening the encoded video on Virtualdub, one little program could easily do this function above and generate a text file with all the freezeframe lines to add it on a AVS script.

I know sometimes it appears one keyframe right after other keyframe, but it's easy to fix manually by watching the video on Virtualdub before encoding.

Does anyone know if it's something possible to do? It would be great, especially for old animes.

[Dreassica]

Just use Yatta to go through the video and manually freezeframe it there and save it into an avs file, eventhough Yatta is kinda awkward to use for that, any automatic attempt prolyl will fuck up at some point, because of scrolling scenes, which obviously will be noticable if u freeze it. Or it might fuck up and see an action scene as a scenechange as well.

[Tanma]

Just use Yatta to go through the video and manually freezeframe it there and save it into an avs file, eventhough Yatta is kinda awkward to use for that, any automatic attempt prolyl will fuck up at some point, because of scrolling scenes, which obviously will be noticable if u freeze it. Or it might fuck up and see an action scene as a scenechange as well.

I had pointed it, but the work for removing some frezeframe lines manually is easier than manually using freezeframe on ~97% of the keyframes.

Using a program like aegisub i can find where are the action scenes, by looking the keyframes through the timing bar.

[Stigma]

Holy thread necromancy batman!! =P

Unfortunately I no logner think a filter like this is possible to make. not because it couldn't be coded, but because that it would either not be effective, or have too many false positives.

the best bet is to use a program to detect keyframes as was sugested... atleast it speeds up the process a lot from going through them all manually.

-Stigma

[Tanma]

Stigma, i think it is something possible.

There is a program that detect keyframes, his name is ymc, that comes with yatta. With the option SCLavc checked, it generates a .yap file with all the keyframe numbers.

Unfortunatelly, I don't know how to program it, but the idea is simple, as we pointed.

For example, I have a text file with the keyframes generated by ymc, that will have something like this:

2486
2692
2725
2765
2830
3165
3374
3636
3731

The solution for avoid problems with action scenes is pretty simple: (A = current keyframe)

if A <= (previous keyframe + 10) then
jump to the next keyframe #or something like that
else
Freezeframe(A-1,A-1,A-2)
FreezeFrame(A,A,A+1)

In some videos (like the one I'm dealing), it is better to use
Freezeframe(A-1,A-2,A-3)
FreezeFrame(A,A+1,A+2)


For example, with keyframes 2486 and 2692, the output would be somethinf like this:

freezeframe(2485,2485,2484)
freezeframe(2486,2486,2487)
freezeframe(2691,2691,2690)
freezeframe(2692,2692,2693)

The best would be if we had a option to insert the number of frames we want to freeze before and after each keyframe, as I told before.

Anyone can do a program to do this stuff?

[Mug Funky]

now this isn't optimal, but "improvesceneswitch" in the removedirt package will replace the frames either side of a scenechange with the next ones along.

so instead of:

1 2 3 4|5 6 7 8

you get:

1 2 3 3|6 6 7 8

this'll get rid of the film splice marks, but in action scenes you'll probably get stutter (DBZ is full of flashing stuff that'll no doubt set off a scenechange detector).

...or you could just leave them in and try not to notice them. the original editor obviously didn't care, so why should we?

you'll notice Bollywood films have this artefact a lot, as do lower budget or older TV shows shot on 16mm.

[Tanma]

this'll get rid of the film splice marks, but in action scenes you'll probably get stutter (DBZ is full of flashing stuff that'll no doubt set off a scenechange detector).

...or you could just leave them in and try not to notice them. the original editor obviously didn't care, so why should we?

you'll notice Bollywood films have this artefact a lot, as do lower budget or older TV shows shot on 16mm.

That's why I had the idea to not freeze a keyframe in a interval =< 10 frames of the previous keyframe. We just need someone to do this script...

[Tanma]

I'm loading removedirt.dll but the script isn't recognizing the command improvesceneswitch...

[MOmonster]

DupHQ is one of my older functions I created for such (similar) problems. I don´t used this function for a long time, but it could be helpful for your problem. It´s worth a try.

[videoFred]

I am looking for a solution for this myself. I'm transfering old 8mm films and at scene change I need to delete -say- 5 frames from the previous scene and 5 frames from the next scene.

Now, Fizicks DepanEstimate is detecting scene change very well. With info=true you will see "base" is the indicator for this.

It must be possible to automize this... I hope?
It would save us a lot of work....

PS: detecting keyframes is not a problem for me.
I capture in uncompressed RGB24 so all frames are keyframes.

Fred.

[cestfait]

I'm pretty sure there's an option for freezeframe at scenechanges in yatta, and you can use pattern guidance and your eyes to modify where the scenechanges are marked. Also, I would use SCXvid.

You can actually do all of the freezeframes yourself by just moving through SCXvid's sections in yatta and freezframing with hotkeys. I've done that before, although the lazier I get, the better I like the automatic option. ;p

[signal]

For older film based anime I tried a simple script to try and possibly identify a good threshold for YDifferenceFromPrevious and/or YDifferenceToNext.
clip=AviSource("my.avi").ConvertToYV12
ScriptClip(clip,"Subtitle(String(YDifferenceFromPrevious)+chr(20)+String(YDifferenceToNext))")

I was getting too many false positives with flashing sequences or in a fairly dark scenes with a brighter character moving around.

There is a consistent pattern of the splice beginning at the bottom of one frame followed by the top of the next frame but
I'm not experienced enough to figure a way on how to use that pattern (with masking?).

Here are two 10MB VOB samples:
test1.vob (http://www.pinesalad.net/example/test1.vob)
test2.vob (http://www.pinesalad.net/example/test2.vob)

In the meantime I did find LBKiller to be a nice manual tool to assist with determining what frames to replace.
Found it less cumbersome than yatta.

It helps that it does a scene change analysis (albeit with false positives as well) but let's you jump to them so you can decide what you want to do with them.

Lbkiller General Procedure Guide.txt (http://esby.free.fr/prog/lbkiller/Lbkiller%20General%20Procedure%20Guide.txt)
lbkiller.21.rc2b.rar (http://esby.free.fr/prog/lbkiller/lbkiller.21.rc2b.rar)

The source and other info can be found at the bottom of this page:
http://esby.free.fr

[Dreassica]

Pretty sure it's feasible to ad some detection to skip flashes via min. number of frames between 2 detected scenechanges ( 1 or 2).
So that it ignores them.

[signal]

Some false positives (not flashes) I'm seeing with YDifference do only last 1 frame.

Some flashes I am seeing also last 1 frame which would lend itself to some false positives.

Seems some kind of comparison of a bottom section of one frame to the top of the next (i.e. not taking the whole frame into account) could yield some useful results when also compared to a full YDifference frame analysis.

Something along the lines of:

If YDifferenceToNext > Threshold
and
Bottom of current frame > Threshold
and
Top of next frame > Threshold

Of course this is specific to the sources I have and may not work for others.

[Zarxrax]

A couple years ago I wrote a program that obtains a list of scene changes by using keyframes in an avi file. The xvid codec is quite good at detecting scene changes, so as long as you compress your video to xvid before running my tool on it, it would produce a quite accurate list of scene changes.

I'll probably be digging my program back out over the next few days to modify it for my own purposes, but while I'm at it, it would be fairly trivial for me to get it to output an avisynth script based on the scene changes.

Would this be useful to anyone? If so, exactly what avisynth commands should it do?

[signal]

Sure! I would like to try that out.
Anything that might save some time on this is welcomed.

With lbkiller I've manually already gone through and created a list of problem frames.

I'd like to compare that frame list to one your app might generate to get an idea of accuracy.

Your primary avs command would be freezeFrame to duplicate previous/next frames.

In my examples I have a splice mark on two sequential frames and need to duplicate the previous frame on the first splice mark and the following frame after the second splice mark.

Example:
Splice Marks on Frames 131 and 132 with freezeFrame would be:

freezeFrame(131,131,130) # replace 131 with previous frame
freezeFrame(132,132,133) # replace 132 with next frame

When lbkiller found a good scene change it was always on the first splice mark. If your detection works the same then the following frame replacement (second splice mark) could be an option.

I'm sure this is still going to take some manual work in the end but narrowing down the scene change frames to something more accurate would be a big plus for me.

[Zarxrax]

Ok, I finished this a lot quicker than I thought. I haven't really tested it at all, so let me know if it doesn't work right, or if it has some off-by-1 errors or something. It requires .net framework 2.0 or higher, I think.
http://amvhell.com/stuff/SceneDetectAVS.zip

[Dreassica]

Chances are it will fail in higjh action scenes and flashes, as keyframes get put there allot as well. I know it does with anime/cartoons.

[signal]

Giving it a shot.
Any particular settings (or version?) for xvid?

[Zarxrax]

Giving it a shot.
Any particular settings (or version?) for xvid?

Nah, nothing in particular. I would just set it up to encode fast, so maybe turn off everything like vhq and bframes and turn on turbo, and stuff. You probably might wanna leave the motion estimation at 5 or 6 though.

[Oursan Pluch']

If that can help you


Function RestoreBadFrames(clip objVideo)
{

Global objClip = objVideo

objVideo = objVideo.ScriptClip( "AnalyzeBadFrames()")
objVideo = objVideo.FrameEvaluate( "global intFrameNumber = current_frame")
objVideo = objVideo.FrameEvaluate( "global fltDiffFromNext = 0.50*YDifferenceToNext(objClip) + 0.25*UDifferenceToNext(objClip) + 0.25*VDifferenceToNext(objClip)")
objVideo = objVideo.FrameEvaluate( "global fltDiffFromPrevious = 0.50*YDifferenceFromPrevious(objClip) + 0.25*UDifferenceFromPrevious(objClip) + 0.25*VDifferenceFromPrevious(objClip)")

Return objVideo
}

Function AnalyzeBadFrames(clip objVideo)
{
objVideo = ( fltDiffFromPrevious < 0.20 && fltDiffFromNext < 0.20 ) ? objVideo.FreezeFrame(intFrameNumber,intFrameNumber,intFrameNumber-1) : objVideo

objVideo = ( fltDiffFromPrevious > 18.0 ) ? objVideo.FreezeFrame(intFrameNumber, intFrameNumber, intFrameNumber + 1) : objVideo
objVideo = ( fltDiffFromNext > 18.0 ) ? objVideo.FreezeFrame(intFrameNumber, intFrameNumber, intFrameNumber - 1) : objVideo

Return objVideo
}


objVideoD = objVideoD.RestoreBadFrames()