YUV colour curves format (+ prelim. filter results)

[mg262]

I'm trying to sling together a filter which will compare two clips and generate curves (like Photoshop curves) which make one look like the other. Some searching suggests that for the RGB mode a sensible curve output format to use is the Photoshop .AMP which can be read by MaskTools RGBLUT -- but I don't know/can't find any natural format to use for the YUV method. Is there any choice which would let me interface with other tools?

Thanks all!

[communist]

IIRC there was some talk in that "Film-Look"-thread about color curves etc. Didée says that there is a color curve plugin that works in YUV for AviSynth. Have a look here:
http://forum.doom9.org/showthread.ph...dll#post592249

You may also want to look at E-Male's GiCoCu (same thread) and BugsBunny's Gradation Curves (for VirtualDub) plugin.
http://forum.doom9.org/showthread.ph...363#post426363

[mg262]

Thank you. Turns out the latest beta of Gradation Curves has a YUV format, so I'll try and make it compatible.

Edit:
Here are some preliminary shots, all in the format:


i.e. the top two pictures are original and the bottom two have been passed through the filter; bottom left is the DVD passed through a curve chosen to make it look as much like the VHS as possible, etc.

YUV colour space, 'mean' method


YUV colour space, 'median' method


All the curves are chosen on a clip-wide basis, not on a per frame basis.

[Mug Funky]

hey, this is a really good idea!

there's been a few occasions where i've had to match one source with another to patch up a glitch in one. this could be really useful for that.

[communist]

Hmm looks nice - though I'm wondering did you copy a DVD to VHS and then captured it - and applied your filter? If yes - doesnt this kinda destroy the point of getting one color style to another source? (I mean if you already have both sources than why make the one look like the other if you could just use the one with the correct 'color style', except when quality is bad).

Or is this just demonstrational - and how good does it work with two different clips. The classic example coming to my mind would be to partly copy film color style to video footage (although both have totally different picture content)?

[mg262]

Originally Posted by Mug Funky

hey, this is a really good idea!

Originally Posted by communist

did you copy a DVD to VHS and then captured it

No. I'm dubbing commercial French DVDs from VHS tapes recorded in Sydney many years ago. The Australian version replaced the French intro with clips taken from scenes scattered through the series, and rather than searching all 52 episodes for these I wrote a tiny 'FindMatch' filter that took a VHS frame:



... and compared it with the DVD, returning a clip made up of the 1% of closest matches...



(That's the first 32 frames from the output. )

As the VHS was blend-resampled, multiply cut, cropped and colour shifted w.r.t. the DVD, I needed Restore24, LineUpScenes, Crop+resize and this colour-equaliser before FindMatch.
_______________

This version assumes the clips are lined up; I was going to include a version that took unrelated clips, but I left it because I had no use for it + I wasn't (previously) sure else anyone was interested. I'll get round to it, though it is annoyingly finicky* so I might be a bit.

*because when you start counting colours of > 4,000,000,000 pixels (= ~10000 frames at DVD res) your counters can overflow if you aren't careful.

[E-Male]

i have the rough algo for such a filter in my head
but no time to implement it and other projects to be finished

also you should know the main problem:
the 2 source-clips must be overlayed perfectly
because the algo will compare pixels with same coordinates from both clips

[mg262]

Originally Posted by E-Male

also you should know the main problem:
the 2 source-clips must be overlayed perfectly
because the algo will compare pixels with same coordinates from both clips

Well, the pictures above are from an example where the original clips aren't lined up in time or space, so you can judge for yourself how much of a problem this is. As noted above, the temporal alignment was done automatically; I do have a filter that can do most of the spatial alignment as a side effect of its main work, but it's far from release state.

IMO the main problem is not lining up the clips but the fact that the colour distortion does not occur purely in the channels of a standard colour space. I.e. to predict the Y of the DVD well we need not only the Y of the VHS but also the U and the V of the VHS. The same is true for RGB mode, which I have also tested (screenshots on request). I could test HSV, but I'm not sure this will produce any better results. I think I know of a way around these problems but it would take quite a lot of work (and a lot more source data than I have!) so I haven't pushed it.

It is also possible to do this in a way that requires no relationship between the two clips -- essentially just taking the histograms of the two (in each channel) and applying a map chosen to make one histogram look like the other. I will try and get round to implementing it, but I've signed up for bit too much at once at the moment!

[E-Male]

>>...the temporal alignment was done automatically; I do have a filter that can do most of the spatial alignment as a side effect of its main work, but it's far from release state.
-can't wait for that filter (if you don't wanna show it publicly, yet, could you PM me with some details; i'd love to have a look at that)
it'll make possible much more uses of filter that use 2 or more clips, like my filter toot

>>IMO the main problem is not lining up the clips..
-well to me it was the most obviouse problem

>>but the fact that the colour distortion does not occur purely in the channels of a standard colour space...
-i think a per-channel RGB or YUV LUT will be fine as a beginning

>>It is also possible to ... essentially just taking the histograms of the two (in each channel) and applying a map chosen to make one histogram look like the other...
-i know almost nothing about histograms (never got to learning about them, yet)
so i can't comment on this approach

[Mug Funky]

... a filter to line up 2 videos with different editing. that would be cool.

for the r4 Astroboy DVDs we had to cut the original japanese to fit the crap-looking american masters, so our release wouldn't look like crap (the US footage was edited down from the jap footage, then copied several times onto poor film stock and had the detail crushed and blurred to a brownish dark slush where a picture should have been. plus it was analog telecine, with so-called "peculiar blending"). this took 8 editors nearly a month for the whole series... and you do it with a filter

[lamer_de]

I would have use for such a filter as well. I have the problem that in one series the first episode has a total different saturation/gamma/etc than the rest of the series, so I'd like to correct that

CU,
lamer_de

[joshbm]

This filter would definately be used by me as well lol...

Regards,
joshbm

[mg262]

This thread was starting to overload my small brain because we were talking about three different kinds of filter (my fault):

Colour matching:
As you've probably seen, I've posted the filter on a separate thread.

Spatial matching:
In the case above I tried using a crop by hand (+ resize of course) with the help of a paint package and Subtract; that worked fine, so I didn't try the automation. @E-Male and others: if you do want to try to automate it, IMO the key point is that once you've have frame-exact temporal matching (inc. cuts) this is essentially just global pan/zoom estimation -- if you wanted to rig it together you could probably script it using Interleave and the text file output of Depan or Deshaker. (Probably wouldn't hurt to colour match the clips first.)

The filter I mentioned will hopefully have some advantages in doing the spatial matching, but it's not just a matter of checking for bugs -- I would have to do a fair amount of coding to tweak it into the form where it could be used for that. (Sorry if I gave the wrong impression -- I did say it was far from release state! It's much more ambitious than anything else I've tried, video wise at least, so I'm keeping it fairly quiet so I don't feel like a complete idiot if it doesn't work...)

One other trick: if you can't get the zoom estimation to work with Interleave, try using the pan estimation on both clips separately and comparing the curves -- one should just be a scaled version of the other, and the scale factor will give you the zoom. Then you can try the interleaved pan estimation separately.

@E-Male - if you do try this + want to bounce ideas off me, or still want a few details about the other filter, feel free to PM me.

Temporal matching:
I did try to deal with this with LineUpScenes, but I didn't get the impression anyone was interested. It may just be that my write-up was too long or too confusing, in which case let me know and I'll try to put something clearer up.

I could have made the alignment process fully automatic, but I didn't because I prefer being able to check things visually -- for example, I've found that sometimes a cut occurs and a fadeout/fadein is placed across the cut, which I would have missed without the visuals. Plus it takes me about 5-10 minutes to line up and check a 20 minute episode, excluding the initial computer time, which is fast enough for me. (Maybe I should also have mentioned that RoughScene can take a while to run, because it needs to scan the whole clip -- so it can seem to freeze.)

@Mug Funky
Well, I'm always willing to work for the salary of seven assistant editors . (Hell, I'd work for the salary of one assistant editor!) But anyway, it doesn't really match up to your plans to replace a half-a-million dollar converter with a filter!