Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd June 2005, 17:23   #1  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
Filter: ColourLike

This was mentioned in a recent thread -- thanks to @communist + everyone else who fed me with ideas, pointers & encouragement.
___________________________________________

ColourLike

Brief description: Makes a clip look like a 'reference' clip by adjusting each colour channel.

Search keywords: colour, color, ColorLike, curve, gamma, levels, channel.

Full description:

WriteHistogram(clip, string outputFile, int every = 1) - computes histograms for each channel of the YV12 clip and stores the results in outputFile. If e.g. every=3 is specified, only every 3rd frame of clip is sampled.

ColourLike(clip source, string sourceHistogramFile, string referenceHistogramFile) - takes a YV12 clip source and histogram files for this clip and a reference clip, and applies curves to each channel, chosen to make source look as much like the reference as possible.

Examples:
Here are some examples from a case where the same clip is present in (unrelated) DVD and VHS sources.

First generate the histograms:

dvd.WriteHistogram("dvd hist 100.txt", 100)
vhs.WriteHistogram("vhs hist 100.txt", 100)


Then, say, adjust the VHS to look as much like the DVD as possible:
vhs.colourlike("vhs hist.txt", "dvd hist.txt")

Or adjust the DVD to look as much like the VHS as possible
dvd.colourlike("dvd hist.txt", "vhs hist.txt")

Here are some screenshot showing the results of both of the above, in the following format:



i.e. the top two pictures are original and the bottom two have been passed through the filter; bottom left is the DVD passed through a curve chosen to make it look as much like the VHS as possible, etc.

YUV colour space


Notes:
- Generating histograms runs at about 75fps on DVD source on my PC (and I think most of that time is MPEG decoding), but this still seems slow because the entire file needs to be scanned. As in the example, you can specify quite a large value of every without affecting the results too much.

- You only need to generate the histograms once, so you probably want the WriteHistogram calls in a separate AVS to ColourLike.

- This filter can be used on completely unrelated clips (which I haven't tried yet); I don't actually think that is a particularly meaningful thing to do, so I'm pretty pessimistic about the results -- but if you do happen to try it, see what the results look like when you just use it on the Y channel, and also when you just use it on the U&V channels. (Use MergeLuma or MergeChroma with source and the output of this filter.)

- For the record, I'm not terribly happy with the results of the filter overall -- as I noted in that other thread, I think there is 'crosstalk' between the colour channels that this process does not account for. So I think a proper colour matching function will have to use something more sophisticated than channel-curves.
mg262 is offline   Reply With Quote
Old 23rd June 2005, 21:48   #2  |  Link
AS
Registered User
 
Join Date: Jan 2002
Posts: 76
Thanks. I am really interested in trying this on totally unrelated materials, e.g. one is bright and other one is very very dark. ^_^
AS is offline   Reply With Quote
Old 24th June 2005, 00:31   #3  |  Link
joshbm
Registered User
 
joshbm's Avatar
 
Join Date: Jun 2003
Location: Land of the Noobs & the Home of the Brave
Posts: 349
Ooooo.... Ahhhh....

Time to do some tests with this. Could one supposedly get a "film" look with this by tricking it?

Ex. Take the DVD footage of a desired film and your DV Cameras footage?

Regards,
joshbm
__________________
Tired of waiting for video encodes? Get a totally new P4 Desktop Computer for FREE.
joshbm is offline   Reply With Quote
Old 24th June 2005, 00:36   #4  |  Link
unskinnyboy
Registered User
 
unskinnyboy's Avatar
 
Join Date: Feb 2004
Location: NTSC R1
Posts: 2,046
@mg262, Shouldn't the name of the file be ColourLike_230506.dll rather than ColourLike_230505.dll? This is June you know, not May.
unskinnyboy is offline   Reply With Quote
Old 24th June 2005, 12:44   #5  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
@AS, @joshbm

When you're applying it to unrelated material, the difference in content can cause a problem. For example think about using The Sound of Music as a reference clip (or even a football game). There's lots of grass and outdoor scenes, so a lot of the video is green -- so the clip you're adjusting will become quite green in places, regardless of it's content. So try to keep the content of the clips as similar as possible. (Also don't make them too short.) Plus as I said try original.colourlike(...).mergechroma(original) (that applies particularly to your case, @AS!)

The first histogram (sourceHistogramFile) doesn't actually have to come from the clip you're using the filter on (a.k.a. source). The curve is generated from the pair of histograms(sourceHistogramFile and referenceHistogramFile), so if you find a pair that work nicely, try using them on other clips. In particular, if you can find a couple of clips with very similar content, one DV and one film, you can try this:

dv.WriteHistogram("dv hist.txt", 100)
film.WriteHistogram("film hist.txt", 100)


Then, on any DV clip:

someotherDVclip.colourlike("dv hist.txt", "film hist.txt")

This is all a bit theoretical as my only sources are DVDs or taped from TV... at the end of the day, try stuff and see if it works -- and put up some screenshots so we can bounce ideas off each other!

@unskinnyboy

Oops. You're totally right, although I would have called it 230605, that being the format we use in England. Maybe I should switch to 23Jun05 to avoid confusion...
mg262 is offline   Reply With Quote
Old 24th June 2005, 14:03   #6  |  Link
ac-chan123
Registered User
 
Join Date: Jan 2004
Posts: 219
Hello,

I have a question aboiut your source. This was "Bell und Sebastian" in germany. But it was only one or two times in TV and never releaed on VHS or DVD. But you have a DVD. Where was that released? Have you the Name of the release and the Company?
ac-chan123 is offline   Reply With Quote
Old 25th June 2005, 09:02   #7  |  Link
AS
Registered User
 
Join Date: Jan 2002
Posts: 76
@mg262

I understand exactly what you mean but that's also precisely reason why I want to do it.
AS is offline   Reply With Quote
Old 25th June 2005, 23:05   #8  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
@ac-chan123 - I've PMed you with all the information.

@AS - in that case, go for it!
mg262 is offline   Reply With Quote
Old 27th June 2005, 11:42   #9  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
the crosstalk issue could be helped by working in HSV space, but there still may be things that it misses. colour-grading (especially digital) can involve some bizarre mixing of channels that would be tremendously difficult to reverse-engineer. as long as the channel mixing is uniform it should be okay, but if for example saturation is increased by subtracting RGB channels, all with different matrices, things could get difficult. also those photoshop colour adjustments that work on lows, mids and highs could be hard to pick from a yuv or rgb histogram.
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 27th June 2005, 12:14   #10  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
I did think about implementing HSV (or HSB, HSL, etc.) but Poynton was so rude about them that I was put off.

What I was thinking of doing was adding a RGB mode to this filter (tedious but straightforward) and implementing a ConvertToHSV which stuffed H into the R channel, S into the G channel, etc. -- so all the standard tools could be used on that colour space. It is an unpleasant hack but probably tolerable in a scripting language.

I'd never heard of colour grading before, but as you say anything which applies operations in more than one colour space will be messy to reverse. I think I do know of a way to attack the problem with a bit of maths but I'm probably not going to follow it up now because I don't have suitable data to test it on -- and also I'm working on way too many filters at once!

Edit: I've just read about the ?standard? colour grading tool Lustre and it uses 32-bit/component colour depth

Last edited by mg262; 27th June 2005 at 12:26.
mg262 is offline   Reply With Quote
Old 28th June 2005, 17:13   #11  |  Link
HighInBC
Registered User
 
HighInBC's Avatar
 
Join Date: Jan 2003
Location: Victoria, BC, Canada
Posts: 144
Thanks, this is a great filter, and I intend to use it often.
HighInBC is offline   Reply With Quote
Old 29th June 2005, 15:20   #12  |  Link
trevlac
budala
 
Join Date: Oct 2003
Location: U.S.
Posts: 545
@mg262

Great idea ... I have some thoughts that may or may not be helpful.

1st - I couldn't find your source or the source of Alexander Nagiller's test YUV curves fitler ... so I'm not sure how you do curves on the U/V. Not sure my knowledge of this matters since you say processing in RGB is much the same.

But Some Ideas ....

- If you process a clip with it's own histogram ... is it different due to any averaging across the entire clip ? Sorry ... I've had no time to really test. I can't run avs on my work machine. Maybe this is a silly question.

- I looked closely at the image of the boy going from VHS to the DVD look. It looks like there are 'outlying' colors (a green/cyan and a purple) in the DVD that are just not there in the VHS. It would be hard to come up with those.

- For cross channel correlation ... maybe you can do multiple color spaces at once. For example, if you do RGB and add luma in the mix, you have a correlation between the RGB channels with the luma.

- As I understand it, the eye is much more sensative to change in brightness vs changes in hue. You might focus on luma. You could even test on B&W images.

- Have you thought about doing some sort of normalization before you compare the curves? For example, setting black level ... setting white level ... and maybe even converting the gamma adjustment back out.

- How about doing mini adjustments on parts of the spectrum? For example, adjust cyan within the range of cyan ... as far as hue and brightness goes.

--------------------
Just some ideas ... frankly ... i think your results look great right now.
trevlac is offline   Reply With Quote
Old 30th June 2005, 14:01   #13  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
@trevlac:

Wow -- lots of ideas Thank you!
Quote:
- If you process a clip with it's own histogram ... is it different due to any averaging across the entire clip ? Sorry ... I've had no time to really test. I can't run avs on my work machine. Maybe this is a silly question.
Not a silly question at all... I really should have explained how the filter worked earlier... I'll post a proper account in another post, but here are the key points.

1) The method used only considers one colour channel. Each colour channel is treated completely separately. So e.g. the U of the output clip depends only on the U of the input clip and the Us of the two clips used to generate the histograms.

2) (In a single colour channel), given two histograms, there is a unique increasing* curve that makes the first one look like the second one, and that curve is picked. What that really means is that if e.g. 4.3% of the reference pixels have a value of U=120, the source is adjusted so exactly 4.3% of the adjusted pixels have a value of U=120. (Of course there are inaccuracies due to rounding and the too-low 8-bit colour depth.)

*By increasing I mean that higher values are always mapped to higher values -- i.e. the curve always keeps going up. This is IMO a very sensible restriction!

Using a clip's own histogram on itself has no effect, because (following the example in 2), with the same clip as source and reference.) the way to make sure exactly 4.3% of the adjusted pixels have a value of U=120 is to leave them alone.
Quote:
- As I understand it, the eye is much more sensative to change in brightness vs changes in hue. You might focus on luma. You could even test on B&W images.
(From 1) above) Y is processed separately to the other channels -- so to the extent that Y is a good representation of luma/brightness, that's kind of happening already. On a B&W clip produced from another B%W clip, the result should be mathematically precise -- i.e. it would reverse any (increasing) curve perfectly. So if you were to take a B&W clip and do any kind of levels/gamma to it, the filter should be able to reverse that perfectly. If it doesn't, either my code is buggy or rounding/condition errors are coming in.
Quote:
- Have you thought about doing some sort of normalization before you compare the curves? For example, setting black level ... setting white level ... and maybe even converting the gamma adjustment back out.
That is a good idea... you could presumably do it by applying Levels to the clips before WriteHistogram and ColourLike? Converting gamma adjustment back out is a bit more complex...I need to generate/post some pictures to talk about this which I'll do a bit later.
Quote:
- I looked closely at the image of the boy going from VHS to the DVD look. It looks like there are 'outlying' colors (a green/cyan and a purple) in the DVD that are just not there in the VHS. It would be hard to come up with those.
You have very sharp eyes. Outliers are generated in each colour channel to the extent that they exist in the reference (because of 2) above), but I think what's happening here is that those points are outliers in both U and V at once -- and while the filter generates outliers in U and outliers in V, they don't necessarily coincide...

Not sure what to do about this, but this and some of your other points are pointing to words HSB (=HSV) or HSL. (Your other two points I'm still thinking about...)

BTW, The source code is in a state that I would feel a little embarrassed about posting, but you're more than welcome to it -- just drop me an e-mail/FTP.
mg262 is offline   Reply With Quote
Old 1st July 2005, 14:34   #14  |  Link
trevlac
budala
 
Join Date: Oct 2003
Location: U.S.
Posts: 545
Quote:
Originally Posted by mg262
Converting gamma adjustment back out is a bit more complex...I need to generate/post some pictures to talk about this which I'll do a bit later.
I really look forward to this. I have been trying to learn this color topic for quite some time.


Quote:
You have very sharp eyes. Outliers are generated in each colour channel to the extent that they exist in the reference (because of 2) above), but I think what's happening here is that those points are outliers in both U and V at once -- and while the filter generates outliers in U and outliers in V, they don't necessarily coincide...
I looked at the pictures on a vectorscope. It was easy to see the different colors there. Below are some pics I made of the boy and the hills. Unfortunately ... because I had to resize things to do a reasonable post ... the 'outliers' are not visible. But in the boy they were actually in the VHS and remained there after your filter. More colors were in the VHS.

Quote:
Not sure what to do about this, but this and some of your other points are pointing to words HSB (=HSV) or HSL. (Your other two points I'm still thinking about...)
Anyway ... I posted these pics because it shows that the shape of the color distribution does not really change. It shifts around (hue) and changes size (sat) ... but the shape remains the same. I think this points to maybe doing an adjustment targeted to each of the specific areas of the spectrum. This might work best as a 2nd pass.







Quote:
BTW, The source code is in a state that I would feel a little embarrassed about posting, but you're more than welcome to it -- just drop me an e-mail/FTP.
I will have to ask for this at some time later. Right now I was just looking so I could see exactly what you were doing. You've been kind enough to explain. I hope to have some time in the future to devote to this.

Last edited by trevlac; 1st July 2005 at 14:38.
trevlac is offline   Reply With Quote
Old 2nd July 2005, 07:59   #15  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
@trevlac, that the vectorscope of yours is awesome. It's going straight into my AVS autoload directory. I missed it when I first came out because I didn't know what a vectorscope was... may be worth putting screenshots on the plug-in's thread?

More soon...

Edit: Okay, I can see Poynton's point:

Y is calculated as 21% red + 72% green + 7% blue

and this is a pretty good measure of luma, i.e. corresponds to what we intuitively think of as luma. In other words, the blue level has hardly any effect on how bright something seems.

Now HSB, HSV, HSL, etc all calculate the brightness/saturation/lightness as

33% red + 33% Green + 33% blue

and this is a horrible measure of luma, because it assumes blue has as much effect as green. So using one of these spaces would throw away all the 'focus on luma' that @trevlac mentions.

Poynton suggests* instead that we hold on to Y as it is, but replace U and V with the angle and length of the vector (U,V). I.e. we use the angle and distance from centre of the coloured points on @trevlac's vectorscope pictures above. This make sense, because you can see that each line going out from the centre (corresponding to an angle) seems to correspond to an intuitive 'colour', and that the further from the centre you get the more vivid colours become.

*Actually Poynton suggests doing this in a colour space called CIELab -- but leave that for now.

Last edited by mg262; 3rd July 2005 at 02:02.
mg262 is offline   Reply With Quote
Old 5th July 2005, 16:08   #16  |  Link
joshbm
Registered User
 
joshbm's Avatar
 
Join Date: Jun 2003
Location: Land of the Noobs & the Home of the Brave
Posts: 349
How do I generate the histogram? I've put in "writeHistogram" and dragged it into VDub it writes an empty file... Please don't tell me I have to render it. I've also tried playing the file in WMP 6 and it gives an exception error to the line where writeHistogram is (but generates a blank file).

Regards,
joshbm
__________________
Tired of waiting for video encodes? Get a totally new P4 Desktop Computer for FREE.

Last edited by joshbm; 5th July 2005 at 16:15.
joshbm is offline   Reply With Quote
Old 5th July 2005, 16:28   #17  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
You don't have to render it, although should be some delay as it scans through the source file. I'm really sorry it's messing up... could you post the script and if possible e-mail me a short section of the source file (I'll PM you with an e-mail address)?
mg262 is offline   Reply With Quote
Old 5th July 2005, 16:32   #18  |  Link
joshbm
Registered User
 
joshbm's Avatar
 
Join Date: Jun 2003
Location: Land of the Noobs & the Home of the Brave
Posts: 349
Code:
AVISource("G:\uvs050630-001.AVI",fourCC="CDVC")

FixBrokenChromaUpsampling()
ReInterpolate411()

WriteHistogram("3ccd.txt", 100)
That is my code. Short and simple.

Regards,
joshbm
__________________
Tired of waiting for video encodes? Get a totally new P4 Desktop Computer for FREE.

Last edited by joshbm; 5th July 2005 at 16:35.
joshbm is offline   Reply With Quote
Old 7th July 2005, 17:33   #19  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
Oops... we sorted this out by PM just after that last post, but I've been useless about updating the thread. It was a colour space problem -- I had carelessly omitted the warning message from the version I released and also my note in the original post that it needed to be YV12 can easily be missed. So - YV12 !

I know I said I wasn't going to keep developing this, but after all the positive comments (thank you!), I did anyway. I added RGB32 support and optional parameters so you can select which planes are processed (and also proper error messages ). I had almost finished adding polar UV support -- essentially using luma, colour and saturation as channels but still working in YV12. I've got a bit tied up but hopefully things will calm down after the weekend...
mg262 is offline   Reply With Quote
Old 31st July 2005, 21:29   #20  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
I'm really sorry this has taken so long -- I put this down and had a mind block on picking it up again.

ColourLike, 31 Jul 05

It now has RGB32 support and also you can write this sort of thing:

vhs.colourlike("vhs hist.txt", "dvd hist.txt", true, false, false)

which means: just process Y (respectively R) but ignore U and V (respectively G and B).

It also has a polar-YUV mode, which as its 3 channels has Y, length of the UV-vector and angle of the UV-vector. Currently the angle is not changed by the filter (see below).

You can use that like this:

dvd
WritePolarHistogram("dvd polar hist 1.txt", 1)
vhs
WritePolarHistogram("vhs polar hist 1.txt", 1)


dvd.polarcolourlike("dvd polar hist 1.txt", "vhs polar hist 1.txt", true, true, true) #last 'true' is ignored

This should in theory change the brightness, contrast, saturation, etc but not affect hue. I haven't found this to be very useful in practice. I think a variant which changed hue would be more useful, but this is ill-defined... or more precisely, because you is represented by a circle rather than a line, there is a whole family of 'hue-equalising' filters...

Be careful to feed the filter with the right kind of histogram... YUV, RGB and Polar-YUV are three different kinds.
__________________
a.k.a. Clouded. Come and help by making sure your favourite AVISynth filters and scripts are listed.

Last edited by mg262; 25th August 2005 at 14:38.
mg262 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:26.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.