Problems using Overlay?

[WaryWolf]

Hello!

In the last week or so i've been trying to produce a reasonable looking version of a video clip. I have two copies of the clip, produced from different sources, and so i'm trying to combine the two to add some details (one version of the video has some field blending, and the other has color bleeding, so i am trying to add the luma part only).

However, after rendering the clip through virtualdub into a huffyuv file, I can see that in some sections of the video, the overlay "de-syncs" and suddenly jumps to another section of the clip, before jumping back. I took a couple of screenshots to help illustrate what i'm talking about:

Overlay working fine:
http://i60.photobucket.com/albums/h25/Warywolf/snapshot20070905125347.jpg

a few frames later, desynchronised:
http://i60.photobucket.com/albums/h25/Warywolf/snapshot20070905125357.jpg

this problem doesn't show up while i'm previewing the clip in virtualdub, although i don't look at large amounts of consecutive frames because the script is quite slow - i normally just skip to different sections. I'm not sure if it is an avisynth- or huffyuv-related problem, or something else completely, but i think the most probable cause is that i've messed up a setting somewhere, so I thought this section would be the best place to post first.

anyway, here is my avisynth script - sorry if it's a bit convoluted!

first.avs:
DirectShowSource("D:\games\video stuff\Superflat Monogram\Superflat Monogram - For Louis Vuitton (Takashi Murakami - Mamoru Hosoda 2003).avi")


Telecide(order=1,guide=2,show=false,post=2,vthresh=25,blend=false)
Decimate(cycle=25,mode=2)

ConvertToRGB()
Crop(14,70,-7,-73)
LanczosResize(640,360)
ConvertToYV12()
AAA(640,360,10,10,1,2,chroma=false)
hqdn3d().fft3dfilter()
FrFun3b(4,4,3)
vmToon(sharpen=false,thinning=0,strength=24,luma_cap=200,threshold=4)


second.avs:
AviSource("D:\games\video stuff\Superflat Monogram\1607.avi")
DeleteFrame(1)
Levels(20, 1, 225, 0, 255)
Crop(0,0,-2,0)
LanczosResize(640,360)
AAA(640,360,10,10,1,2,chroma=false)
hqdn3d().fft3dfilter()
FrFun3b(4,4,3)
vmToon(sharpen=false,thinning=0,strength=24,luma_cap=200,threshold=4)

overlay.avs:
goodclip = Import("D:\games\video stuff\Superflat Monogram\first.avs")
badclip = Import("D:\games\video stuff\Superflat Monogram\second.avs")


Overlay(goodclip, badclip, mode="luma", opacity=0.5)

please help, if you can!

[grannyGeek]

( I am a novice, but until someone who really knows comes along . . . . .

Your first script uses Telecide and Decimate.
Your second script for the second clip does not.

Are the two resulting clips at the same frame rate? If not, that might be causing a problem.

If you want to overlay two clips to blend them, all the frames must line up, or you will get "double-exposure" such as shown in your screen shot.

Before you do the overlay, open each clip in a separate instance of VDub or other player, and make sure that frames at various intervals match.

[WaryWolf]

thanks for the reply, grannyGeek. I've probably made a novice mistake, so it would be fitting that a novice could see it...

I used deleteframe (and crop) to make sure that both clips were on the same frame, and everything seemed to line up in virtualdub while i was seeking through, but i just checked the framerate, and it seems the two clips had very slightly different framerates. I just added a quick AssumeFPS to each script, and i'll re-render the clip now, hopefully this will fix the problem.

P.S.: i uploaded a short sample (5 seconds, 850kb, xvid encoded) of the problem, since it's kind of difficult to describe. as you can see, the de-synchronisation happens suddenly, and then jumps back into place.

Link (http://rapidshare.com/files/53520321/superflat-sample.avi.html)

[grannyGeek]

dang it, I won't be able to look at your clip till I get home tonight, this computer doesn't have Xvid or Divx installed.

If the AssumeFPS doesn't help (and it did NOT work for me when I was trying to splice 2 clips with slightly different frame-rates) --- if the un-synch always happens at the same spot, try using Trim to break the clips into segments at the frame where the desynch happens, and Splice them back together, see if that helps. For a test, just Overlay the problem segments by themselves, and see if they will line up correctly.

[IanB]

@WaryWolf,

First some performance tips...ConvertToRGB()
Crop(14,70,-7,-73)
LanczosResize(640,360)
ConvertToYV12()Colour space conversions are slow and lose quality. Okay YV12 won't let you crop by odd amounts, but the resizer core will, in fact it will allow fractional (sub-pixel) amounts.LanczosResize(640,360, 14,70,-7,-73)Will achieve almost the same result without the 2 colour space conversions and can crop by odd amounts.

MergeLuma(goodclip, badclip, 0.5)With opacity=0.0, 0.5 and 1.0 special extremely fast ISSE code is used, for other values fast'ish general purpose MMX blending code is used.

There is a subtle difference between filtering the individual streams and only filtering the combined output stream only.AAA(640,360,10,10,1,2,chroma=false)
hqdn3d().fft3dfilter()
FrFun3b(4,4,3)
vmToon(sharpen=false,thinning=0,strength=24,luma_cap=200,threshold=4)You maybe able to apply some or all of these filters to the resulting merged stream, with little or no difference and a hugh increase in speed.


Now to your problem :- Many DirectShow splitters, codec and filters do not support frame accurate seeking. Normally .AVI files do not have this problem, however you do have a problem.DirectShowSource("D:\g... Hosoda 2003).avi")

Telecide(order=1,guide=2,show=false,post=2,vthresh=25,blend=false)
Decimate(cycle=25,mode=2)If this code sequence is returning different frames on subsequent accesses, you can try setting SetMemoryMax() to a high value and hope with sufficient cache that any small amount of out of order access is serviced thru the cache. If this fails then run this portion of the script out to a temporary workfile using a lossless keyframe format, like huffyuv, to avoid any seeking related difficulties.

And of course you should check that your Telcide and Decimate parameters are correct. Comment all the filters out and replace the Overlay/MergeLuma with a simple StackHorizontal. Examime the resulting output carefully to make sure the 2 streams actually remain in sync all the time. Without the slow filters just comparing the frames should be very fast. You can also test for problems seeking this way by simply scrubing the timeline in VirtualDub to random points.

One issue that may cause problems like this is dropped frames in either stream, these will cause a single frame aberation. If you have these you will need to code your script to explicitly resolve any such problems.

[WaryWolf]

IanB: thanks for the lengthy reply!

i know the script is a bit clumsy, so thanks for pointing out the improvements i can make! :-)

I thought there might be something that would allow me to crop odd amounts with YV12 colourspace, but i didn't look into that. now the script is running a bit faster (and hopefully the quality will be affected).

I had also seen (and tried) the mergeluma filter, but i wasn't sure if there was any difference between that and overlay. If it's faster, then i'll give it a go, thanks.

I considered using deblocking/smoothing after merging the clips, in fact those lines are in overlay.avs but commented out, but since the clip is only ~7000 frames, and i am aiming for a really high quality result, i decided to process them seperately. I know that more processing time does not necessarily mean a better outcome, but after trying both orders I think processing seperately gives a better look to the video.

as for the suggestion about out of order frame seeking, i'm not sure that this is the problem, because when i skip through the video (using alt-G in virtualdub to go to a specific frame number), the video looks fine, these errors only show up while processing the clip continuously, when i'm encoding to huffyuv.

But I think grannyGeek has already found the root of the problem in the different framerates, because I finished a test encode of the clip without any smoothers on (to save time), and the result shows none of this "de-syncing" problem that i was having. I'm encoding it again with smoothing filters on, just in case it was one of the smoothers causing the problem, and if so, then I will definitely try your suggestion.

[Didée]

@ WaryWolf:

Do not use Telecide().Decimate(x) on fieldblended animation. This cannot be successful.

Use MRestore from MOmonster's R_pack (http://forum.doom9.org/showthread.php?t=95924).