Auto-alignment?

[RidgeShark]

Is there a filter for avisynth that works similarly to the auto-alignment feature in Photoshop? I'd like to take 2 video transfers of the same movie, and have one aligned to be nearly identical to the corresponding frame of the other. My intended purpose for this is to copy the chroma from one version of a film to another. Here's an example -

VidSource1 -
http://img132.imageshack.us/img132/4802/kongbw.jpg

VidSource2 -
http://img10.imageshack.us/img10/8691/kongcolor.jpg

Auto-aligned, merged-chroma, and luma adjustment -
http://img580.imageshack.us/img580/9492/kingkongfinalcolor1.jpg


I've looked around and nothing's turned up. Is there anything like this yet for avisynth?

[Guest]

Wouldn't it be easier to just buy the BluRay?

[um3k]

I imagine you could do it with DepanInterleave and a wee bit of trickery.

[RidgeShark]

Wouldn't it be easier to just buy the BluRay?

Well this was just a fun example. And as far as I know, the 80's colorization of King Kong will never come out on Blu-ray. I'd certainly buy it if it did though. I own two VHS copies and the UK DVD (itself a NTSC->PAL transfer from an old 80's video master) of the colorized version. The quality on those are nowhere near the recently restored black and white versions though, which is just a part of the fun of this little experiment.

A filter like this would be interesting because manually aligning two separate film transfers like these is nearly impossible. There's so many little differences from different telecine equipment that a manual alignment would have to be done by sequence and sometimes frame by frame, and would take ages to do. By automatically aligning the two sources, a great deal could be done with both the luma and chroma channels.

[Guest]

Fair enough.

I'm not aware of any Avisynth filters that will directly do that.

[Jenyok]

Will try this...
May be help to you...


ColorYUV2(opt="coring", showyuv=false, analyze=false, autowhite=true, autogain=true)

[WorBry]

I've not actually tried it, but apparently you can use the (absolute) Difference blend mode in Overlay to 'manually' align images with identical visual content. When perfectly aligned, the content outline should turn black. I also included a Difference blend mode in the (YV12 based) Blend_MT function that I put together.

http://forum.doom9.org/showthread.php?p=1400434#post1400434

Again, never tried it in this context. I dont know of an 'auto' equivalent.

[pbristow]

Damn! I just wrote you a long detailed set of suggestions, and the damn forum engine swallowed it 'cos it decided I wasn't logged in anymore! :( (And just for once, I forgot to copy-paste it all into notepad before submitting it... )

If this post works OK, then I'll write it all out again.

[Gser]

Why not just do it in Photoshop? It supports video.

[pbristow]

OK, that worked, let's try again.

Based on WorBry's suggestion, you *could* try using a chain of "?" and ":" (as a "virtual case statement") to compare the images in several different positions, and thus select the offset that works best. You can wrap that into a run-time function and evaluate it agains each frame (or pair of frames). Two problems with that: One is that the chain of comparisons will get very long and run very slowly; the other is that it won't take into account any geometry distortions within the frame. For that, you'd have to split the frame into blocks or regions (with some overlap), compare and adjust them separately, and then merge them back together.

Luckily, there's a toolset that already does these kind of things. We can "misuse" MVtools2. :)
- First, make the two versions identical in size, with approximately matched positions, and starting on the same frame.
- Interleave the two streams, to create a clip with colour and monochrome pairs of frames.
- Use MAnalyse (with chroma=false) to compare each frame with the one before.
- Use Mcompensate or MFlow to shift blocks/pixels from the colour frames to exactly match the positions in the monochrome frames.
- Use SelectEVen or SelectOdd to pull out just the shifted colour frames from the result.
- Use MergeChroma to add the shifted chroma to the untouched monochrome source.

Good luck! :)

[Gavino]

My FindFrame function from this thread, or something like it, might be useful.
Damn! I just wrote you a long detailed set of suggestions, and the damn forum engine swallowed it 'cos it decided I wasn't logged in anymore!
After logging in, use the back button on your browser to get back to the text you had.

[pbristow]

My FindFrame function from this thread, or something like it, might be useful.

After logging in, use the back button on your browser to get back to the text you had.


That doesn't always work, and didn't this time (I just got a completely blank input field).
No prob though, I think I actually did a better job of explaining things second time anyway. :)

[RidgeShark]

Thank you all for the suggestions. I've tried them all but with no usable results, though I did have some interesting results with pbristow's suggestion. The results were very blocky, but the color source did accurately align with the black and white source - however there were missing colors and lots of color fluctuation. I see a lot of potential with this, but I really don't know how to tweak it to perfection.

If anyone wants to help me experiment, here are two short clips that are frame-matched - http://www.megaupload.com/?d=GI47Y5TD

[um3k]

Try this:

col = AVISource("C:\Documents and Settings\Justin (um3k)\My Documents\Downloads\experiment\SD.avi")
bw = AVISource("C:\Documents and Settings\Justin (um3k)\My Documents\Downloads\experiment\HD.avi")

colw = col.Width()
colh = col.Height()
bww = bw.Width()
bwh = bw.Height()
borw = (bww-colw)/2
borh = (bwh-colh)/2
col = col.Lanczos4Resize(1100, 756)
col = col.AddBorders((1280-col.Width)/2, (960-col.Height)/2, (1280-col.Width)/2, (960-col.Height)/2, color=$000000)
bwp = bw.AddBorders((1280-bw.Width)/2, (960-bw.Height)/2, (1280-bw.Width)/2, (960-bw.Height)/2, color=$000000)
int = Interleave(bwp,col).ConvertToYV12()
est = int.Greyscale().depanestimate(range=1,pixaspect=1.0,zoommax=1,improve=false,trust=0)
dep = int.depaninterleave(est,pixaspect=1.0,prev=0,next=1,subpixel=2,mirror=0)
acol = dep.SelectEvery(4, 1).Crop(144, 120, 992, 720)
MergeChroma(bw, acol)

It works pretty well on the sample footage, and should work fine on the rest as long as the relative scale remains constant.

EDIT: Of course, replace the clips with the full movie.

[um3k]

RidgeShark, have you tried my script? I'm curious to know whether it worked for the whole video.

[RidgeShark]

I've tried it on a few small sections and it seems to work great. ;) I haven't been able to do the whole film due to lack of time for all the necessary processing.

My copy of the colorized film is from a PAL DVD that was created from an improper NTSC->PAL conversion, so I used Srestore to create 23.976FPS progressive clips to do these tests. I need to apply it to the whole colourized version and then go through both the color and b&w copies and correct any points where they go out of sync. The restored b&w version has some shots that are longer in length by a few frames than the colorized version, which throws everything off.

And being that I am not a very advanced avisynth user, could you possibly explain how your script works? I basically understand it, but not enough to modify it for other videos. Thank you so much for your help. :)

[Mug Funky]

judicious resizing, interleave(a,b) and mvtools will get you there.

i've done something similar with using mvtools to fix a very old (rank mkII maybe?) transfer that had inter-field wobbling such that the picture shook around unbearably.

only gotcha is that you need to make sure both clips are frame-for-frame the same edit, or it'll go pear shaped.