Conversion deblending/chroma deblending using reference source - unstable test filter

[wonkey_monkey]

Hi all,

I've written a filter to restore video with blended fields/frames using a reference source - in my case, I have an SD source at native 25p and an HD source of the same video which was converted to 60i.

srestore did a near-perfect job; however, I wasn't happy that srestore doesn't give consistent results when seeking, nor that I still had residual chroma blends.

So I wrote a couple of very quick filters, one to pick out the unblended frames from the (bobbed) HD video to match the SD video, and another (possibly the more useful) to undo any residual chroma blending.

Here's the download: specials.zip (http://horman.net/specials.zip) (caution: potentially highly unstable!)

And here's a very rough overview:


blended=... # a blended HD source, in my case a Blu-ray rip
reference=... # an SD source of the same video, in my case a DVD rip

# now prepare two matching progressive clips from the above
# (I cropped because the DVD rip had black bars at left and right but the Blu-ray didn't;
# I also had to shift one picture a little, and blurring both may help get saner matches.
# You may also have to do some trimming)

blended_compare=blended.bob.bicubicresize(720,576).crop(16,0,-16,0).blur(1.58)
reference_compare=reference.crop(16,0,-16,0).blur(1.58)

# now use the filter planetmatch to restore the unblended frames

unblended=planetmatch(blended, blended_compare, reference_compare)

#########################
# if there is still blended chroma:
#########################

unblended_compare=unblended.bicubicresize(720,576).crop(16,0,-16,0)
# ^ to match reference_compare from above

chromafix(unblended, unblended_compare, reference_compare)



Please note: the filters should be used with caution - they do absolutely no sanity checking, and have only been tested with YV12 clips. If there is interest I will tidy them up, but I want to gauge that first as it seems an unlikely scenario.

Sorry if my terminology is a little confused - different kinds of blends being discussed with the same terms, etc - and for the cryptic filter name :)

Edit: chromafix in action: (image) (http://horman.net/chromafix.jpg)

David

[StainlessS]

but I want to gauge that first as it seems an unlikely scenario

Dont think at all unlikely, I'm sure a number of people would be interested in your ideas in this respect
and would love to have a gander at the source (even if you did not tidy it up).

I had an idea of doing a plugin version of recent MatchFrames script to do a single scan storing eg aveluma,
minluma, maxluma, in arrays (for the hires c clip) and only doing compares with lores f clip frames where
the stored luma values were a close approximation to that of the f clip find frames. Dont think I'll bother doing
this but it could speed up detection somewhat.
EDIT: But probably not much use to you in PlanetMatch.

[wonkey_monkey]

Here's the untidy source: http://horman.net/specials-src.zip. There's an unintentional pun on "special sauce" in there somewhere...

planetmatch uses the length of both comparison clips to work out the approximate position of each unblended frame, then checks a couple of frames either side and returns the best luma match (it also, possibly redundantly, checks chroma either side of the best luma match in case there is a better match there - I think this would now be taken care of by chromafix anyway).

chromafix is run on the result from planetmatch (i.e. it should match the reference clip in luma at least), and compares the chroma from the previous, current and next frames against the reference source frame, and if the current frame is the best match it then calculates and finds the best match among some "unblends" - current*1.x-previous*x and current*1.x-next*x, where x ranges from 0 to 0.5.

(FYI, palrestore is a filter for restoring a different, specific case of 60i to 25p)

David

[pbristow]

I've written a filter to restore video with blended fields/frames using a reference source
...and you published it on my birthday! :)

I shall look forward to examining your code soon, if I can find enough free time.

[StainlessS]

Thankyou for the source. :)

[resonator79]

< Off Topic >
I'm attempting to fix my DW-Specials BD, and I ran across this thread, as well as another one with some script examples.
David, I was able to use your experimental plugin successfully on The Next Doctor and The Waters of Mars so far, INCREDIBLE work on your parts, worked like a charm. I was able to fix the remaining interlaced parts nicely, and I fixed the credits, differently though, since the DVDs I got are terribly blocky, so I used AE Pixel remapping from a (bobbed) 60p.
< / Off Topic>

So first of all, thanks!

Now, I'm trying to use planetmatch as well, however, I can't quite figure it out.

I used:


blended=AVISource("dw_straight_bd.avi").ConvertToYV12()
reference=AVISource("dw_straight_dvd_synced2bd.avi").ConvertToYV12()


blended_compare=blended.bob().bicubicresize(720,576).blur(1.58)
reference_compare=reference.blur(1.58)

unblended=planetmatch(blended, blended_compare, reference_compare)

unblended_compare=unblended.bicubicresize(720,576)

chromafix(unblended, unblended_compare, reference_compare)



I've synced up the DVD temp-decode AVI to be frame-accurate (as much as that's possible with 29,97i and 25p sources) in advance, just to make sure that they are and stay in sync.
I removed the cropping, because the DVD version I have didn't have any borders.

Yet, the output is clearly interlaced and has chroma-shifting going on as well...

Am I missing something? Could it be an issue with certain other plugins, a wrong version or something?

[wonkey_monkey]

I got your PM, but have currently mislaid my scripts. But from what you've now posted, I think I can see what you've missed - blended needs to be bobbed, not just blended_compare. I used QTGMC by itself to write a losslessly-compressed bobbed version, then used that as my input for the matching stage.

You should also ensure that your two compares line up as closely as possible, spatially (as well as temporally), otherwise you'll end up getting the wrong frames on slow pans. Use a frame from the start of a shot and modify bicubicresize parameters to shift.

[resonator79]

Will try that ASAP. Thank you!

[resonator79]

Unfortunatly, after having spent a few hours trying, I gave up, couldn't get it to work.
However, like you said in the other thread, srestore works pretty well, it's just scene changes that are a gamble.
I used Torchwood HD-DVD for this example, but Planet Special is pretty much the same:

http://artistdead.com/pics/torch_blend.jpg

Now, with the Planet of the Dead special, I could go through in Premiere Pro and substitute the frames with the clean frames from the bobbed file, but that's a lot of work, and I'd really like to fix my Torchwood HD-DVDs as well, so...

I've been thinking about modifying srestore to use an override file, but I don't know how. Do you have any ideas how to fix this?