Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th April 2007, 20:57   #1  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Linear interpolation of color correction with key frames

I try create a new plugin (beta version) ColorKeyFrame - Linear interpolation of color correction with key frames.

(It is useful for clips with varying clopors, for example for DV camcorder clips in complex light condition)

The plugin is similar and based on ColorYUV AviSynth filter.
In Avisynth it is a hard to set different color correction for various part of clip, especially smoothly varying correction.
(the Animate function is not very convinient).
There are two functions in new plugin: ColorKeyFrame and ColorInterpolate.
Several ColorKeyFrame commands can be used to define color correction data for some given frames (key frames).
ColorInterpolate function make color correction by given values at key frames
and smooth transition of color correction between key frames by linear interpolation of the color correction data.
If you define exactly one key frame, the color correction by ColorInterpolation
will be uniform for whole clip, just as ColorYUV.

First version of the plugin uses some hack of Avisynth interface to transmit its setting
(the similar method is used by DGDecode (hints), IVTC, DePan, MVTools (motion vectors), etc).
Output of ColorKeyFrame function is special clip with color correction data
(for given key frame and previous key frames) in framebuffer.
This output can be used as a source for next call of ColoKeyFrame
for defining of correction of some other key frame, and so on.
Finally the output of last ColorKeyFrame is used as second input clip (keys)
for ColorInterpolation function to interpolate the color correction.

Now all frames store the same data (it is not per-frame property).
May be more apppropriate method is exist for storing and transmitting of per-clip data?
Firstly I supposed that user_data in plugininit2 function may be used.
Or may be strings?
(Text file may be used too)
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 19th April 2007 at 18:01.
Fizick is offline   Reply With Quote
Old 18th April 2007, 22:39   #2  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
Just a thought (maybe wrong):

Since both filters are inside one dll, you can put the ColocKeyFrame data in an STL container (a map) and use child's address as the key. Because many ColorKeyFrame calls may be chained to a single clip, the value of the map should be a container also (a stack, a queue, a vector, a list,..., depending on how ColorInterpolation accesses the data).

ColocKeyFrame pseudocode:
Code:
your_type data = CalcData(child, keyframe);
size_t key = (size_t) child->p;
map<size_t, list<your_type>>::iterator it = global_map.find(key);
if (it == global_map.end()) {
    // first time; init value container
    global_map[key] = new list<your_type>(1, data); 
}
else {
    // not first time; add an element
    it->second.push_back(data);
}
ColocInterpolation pseudocode:
Code:
size_t key = (size_t) second_child->p;
map<size_t, list<your_type>>::iterator it = global_map.find(key);
if (it == global_map.end()) {
    // no values; handle error
    ...
}
else {
    // it->second contains the list with data; do your stuff
    ...
}
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 19th April 2007, 17:51   #3  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
I do not familiar with C++
what if we have several clips in one script to tune the colors?


Probably we must add some option to avisynth 2.6 (2.7) for per-clip metadata?
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 20th April 2007 at 03:41. Reason: one script (not clip)
Fizick is offline   Reply With Quote
Old 20th April 2007, 01:54   #4  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
Damn, this forces me to think more . Can you perhaps supply the argument lists of ColorKeyFrame and ColorInterpolation? A bit more of information would certainly help. Up to this point, as I re-read more carefully your 1st post's description, I can only conclude (hopefully correct) that either:
  • you seek a linearly changing ColorYUV on any of its argument per frame, based on values on frames f0, f1, f2, ..., or
  • something different, which however needs only a single-frame clip to hold the info needed by ColorInterpolation.

In the first case you could (since STL is not a choice) setup
  • a structure to hold child->p, f0, f1, f2, ..., arg10, arg11, arg12, ..., ..., argn0, argn1, argn2, ..., (except for the first all others should be lists)
  • a linked list of those structs to identify the struct associated with its first argument (the source clip) with a linear (in size) search
Then you could (in each step a scan of the linked list for a match on child->p is implied)
  • In each ColorKeyFrame, insert a new "column" to the struct that corresponds to clip passed in (ie add an item in each list)
  • In ColorInterpolation::GetFrame construct for each k=1,...,n an f0, argk0, f1, argk1, ... array to pass to Spline (if up to ~25 keyframes are enough) or your linear interpolation function (if > ~25 keyframes are wanted), gather all calculated values and make the final env->Invoke to ColorYUV.

In the second case ColorKeyFrame could simply return this clip as its value and the user would need to supply it as an optional arg in each succesive ColorKeyFrame call. If the above is however considered complicated for the user (probably is) then the following recipy could be used (ColorKeyFrame returns the source clip):
  1. In the constructor of ColorKeyFrame, env->GetVar for a global var that is based on child->p value as a hex string (plus a long prefix to reduce the posibility of conflicting with user script variables). Example: __color_key_frame_0x2E48AB0C
  2. If var is found and is a clip, update it (we are inside a succesive call to ColorKeyFrame); if not create it (we are inside the 1st call for this IClip*)
  3. In the constructor of ColorInterpolation, env->GetVar for the above global var; if found proceed else exit with error

Another approach that came to me while writing those lines - but which did *not* devoted much time to think of - would be to:
  1. make a base filter class that holds a private frame buffer (initialised to child->GetFrame(0) ?) and returns it for every value of n in its GetFrame.
  2. derive ColorKeyFrame from it to modify appropriately the frame buffer based on the other arguments
That way we modify a single frame buffer only at the constructor, irrespectively of number of frames.

I will stop here because my head started complaining for scratching it too hard ; hope something of the above is of use.
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 21st April 2007, 10:15   #5  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
gzarkadas,
I wonder what information do you ask?
You can read a source code!

Thabks for discussion.
Firsly I wanted to use NewVideoFrame and GetFrame in constructor.
I also try to use only frame 0. But it does not give any speed increasing.
Some more discuss later.

Anyway, I released version 0.2 (21.04.2007)
Fixed correction before first and after last key frames.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 21st April 2007, 10:36   #6  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
Quote:
Originally Posted by Fizick View Post
I wonder what information do you ask?
You can read a source code!
Well, I missed that one; I thought it was still unpublished. Downloaded and will start reading soon .
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 21st April 2007, 15:47   #7  |  Link
gzarkadas
Registered User
 
gzarkadas's Avatar
 
Join Date: Sep 2005
Location: 100011110010001000001 10000011111111000001
Posts: 222
I have read it and from what I see it is a linear ColorYUV interpolator (case 1 from my 2nd post).

IMHO there are two options, to improve it:

Option 1

Make ColorKeyFrame a simple function that returns a formatted string by applying sprintf to its arguments. No need to pass a clip in, since it is only used as a data container. Say (AviSynth syntax):

Code:
ColorKeyFrame(int frame, float gain_y, float off_y, float gamma_y, \
float cont_y, float gain_u, float off_u, float gamma_u, float cont_u, \
float gain_v, float off_v, float gamma_v, float cont_v)
Modify ColorInterpolation to accept an arbitrary number of ColorKeyFrame-formatted strings as additional arguments ("cs+" as string to env->AddFunction). Unpack then the strings into the filter's constructor, construct an array of KeyData structs and store it as a class member.

That way in GetFrame you only need to make the interpolation from internal-to-the-filter data; everything else has been done inside the constructor.
Option 2

Make your own data management as I have described in my previous posts (with STL containers preferably, else you will have to low-level manipulate data structures yourself). It is more elaborate but it does not being limited to 59 keyframes as Option 1 above (due to 60 args limit of AviSynth).
IMO the <= 59 keyframes limitation of Option 1 is not that serious (one can always Trim the source clip and apply ColorInterpolation part by part), since most of the time the number of keyframes will be limited; I would prefer it since it is a lot simpler. It is easy also to call the filter from the script either by assigning to intermediate string vars or by using the following example construct:

Code:
ColorInterpolation(clp, 
  \ ColorKeyFrame(38, gain_y=295,  gamma_v=1.1),
  \ ColorKeyFrame(235, gamma_y=0.95,  gamma_v=1.1),
  ...
  \ ColorKeyFrame(12120, gain_y=427,  gamma_u=0.89) )
If however you prefer the 2nd option, I could try (in about a month from now) build an STL container-based implementation of keydata.
__________________
AVSLib, a free extension library for Avisynth. Current version: 1.1.0 (beta), 14/05/2007.
[ Home page | Download page ]
gzarkadas is offline   Reply With Quote
Old 21st April 2007, 17:08   #8  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
I understand your suggestion.
It may be done this way(s).
But I am not sure what is better (faster) and simpler - too many agruments.
More comments from other users (and developers) is needed.

Right now I encode my worst DV video I ever shoot,
in dark cafe and outdoor, with varying color balance, gain, and huge noise in dark.
I create this filter specifilally for this video and I am quite happy with current syntax.
Here is a script (encoding speed to MPEG2 DVD is 0.725 fps).
I use AvsP program for editing (copy-paste), frames number insert, and preview. We also can use sliders, but it is not very confinient (they all must be renamed after copying of line)
Code:
avisource("F:\capture\september\jubiley.06-09-03_19-59.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-00.00.avi")
keys1=colorkeyframe(0,off_y=16,gain_y=-36,off_u=0,off_v=0)
keys1=colorkeyframe(keys1,749,off_y=16,gain_y=-36,off_u=0,off_v=0)
keys1=colorkeyframe(keys1,812,off_y=6,gain_y=0,off_u=0,off_v=0)
keys1=colorkeyframe(keys1,840,off_y=0,gain_y=150,off_u=8,off_v=-16)
keys1=colorkeyframe(keys1,946,off_y=0,gain_y=150,off_u=8,off_v=-16)
keys1=colorkeyframe(keys1,988,off_y=0,gain_y=150,off_u=16,off_v=-22)
keys1=colorkeyframe(keys1,1115,off_y=0,gain_y=150,off_u=16,off_v=-22)
keys1=colorkeyframe(keys1,1125,off_y=0,gain_y=170,off_u=16,off_v=-22)
keys1=colorkeyframe(keys1,1247,off_y=0,gain_y=170,off_u=16,off_v=-22)
keys1=colorkeyframe(keys1,1277,off_y=0,gain_y=170,off_u=16,off_v=-26)
street=ColorInterpolate(last,keys1)
#return street.coloryuv(analyze=true) 
#cafe normal light
avisource("F:\capture\september\jubiley.06-09-03_20-02.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-03.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-03.01.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-06.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-07.00.avi") 
keys2=colorkeyframe(0,off_y=0,gain_y=40,off_u=10,off_v=-15)
keys2=colorkeyframe(keys2,1802,off_y=10,gain_y=130,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,2059,off_y=0,gain_y=130,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,2579,off_y=0,gain_y=130,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,2683,off_y=16,gain_y=180,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,2736,off_y=10,gain_y=180,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,2736,off_y=16,gain_y=220,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,3737,off_y=16,gain_y=220,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,3775,off_y=0,gain_y=100,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,4044,off_y=0,gain_y=100,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,4053,off_y=16,gain_y=140,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,4071,off_y=16,gain_y=140,off_u=4,off_v=-8)
keys2=colorkeyframe(keys2,4097,off_y=0,gain_y=120,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,4627,off_y=0,gain_y=120,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,4645,off_y=10,gain_y=150,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,4965,off_y=10,gain_y=150,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,4980,off_y=0,gain_y=130,off_u=6,off_v=-13)
keys2=colorkeyframe(keys2,5489,off_y=0,gain_y=130,off_u=6,off_v=-13)
keys2=colorkeyframe(keys2,5513,off_y=16,gain_y=190,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,5650,off_y=16,gain_y=190,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,5664,off_y=16,gain_y=270,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,5738,off_y=16,gain_y=270,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,5754,off_y=0,gain_y=200,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,6070,off_y=0,gain_y=250,off_u=6,off_v=-12)
keys2=colorkeyframe(keys2,6529,off_y=0,gain_y=250,off_u=6,off_v=-12)
cafenormal=ColorInterpolate(last,keys2)
#return last.coloryuv(analyze=true)
#cafe dark
avisource("F:\capture\september\jubiley.06-09-03_20-27.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-30.00.avi").trim(78,0) ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-43.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-44.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_20-55.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_21-08.00.avi") ++ \
avisource("F:\capture\september\jubiley.06-09-03_21-09.00.avi").trim(690,0) ++ \
avisource("F:\capture\september\jubiley.06-09-03_21-10.00.avi")
keys3=colorkeyframe(0,off_y=0,gain_y=180,off_u=5,off_v=-6)
cafedark=ColorInterpolate(last,keys3)
#return cafedark.coloryuv(analyze=true)

street ++ cafenormal ++ cafedark
#return last
converttoyv12(true)

separatefields()
i=last
d=degrainmedian(i,interlaced=true,mode=0)
vf1=mvanalyse(d,pel=2,blksize=8,isb=false,idx=1,overlap=4,sharp=2,delta=1)
vf=mvanalyse(d,pel=2,blksize=8,isb=false,idx=1,overlap=4,sharp=2,delta=2)
vb=mvanalyse(d,pel=2,blksize=8,isb=true,idx=1,overlap=4,sharp=2,delta=2)
vf2=mvanalyse(d,pel=2,blksize=8,isb=false,idx=1,overlap=4,sharp=2,delta=4)
vb2=mvanalyse(d,pel=2,blksize=8,isb=true,idx=1,overlap=4,sharp=2,delta=4)
mvdegrain2(i,vb2,vb,vf,vf2,thsad=600,idx=2,thSCD1=600)
#return last.weave
g=mvdepan(last,vf1)
depanstabilize(data=g,cutoff=3,mirror=15,blur=30)
#return last.weave
weave
fft3dgpu(sigma=1.5,interlaced=true,plane=4,bt=3,ow=16,oh=16)
#return last
#stackvertical(i,last)

limiter()
converttoyuy2(true)
I also believe, that key frames approach may be useful for denoiser too. I can add "sigma" parameter to ColorKeyFrames and use it in new (modified) version of fft3dfilter (or other denoiser).
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 24th September 2010, 13:42   #9  |  Link
cobo
Registered User
 
Join Date: Mar 2004
Posts: 118
ColorKeyFrame only works if I use whole numbers in the variables, not with decimal numbers.

This gives the expected result -> ColorKeyFrame(217,off_u=-0, off_v=8)
This doesn't change the clip at all -> ColorKeyFrame(217,off_u=-0.25, off_v=8.25)

Is it possible to make it work with decimals?

Last edited by cobo; 24th September 2010 at 13:47.
cobo is offline   Reply With Quote
Old 25th September 2010, 19:22   #10  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Probably yes for input. But correction will be integer anyway.
I wonder, that clip is not changed. Will look to code.

EDIT: the bug is found. Will release fixed version today.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 25th September 2010 at 20:01.
Fizick is offline   Reply With Quote
Old 26th September 2010, 20:05   #11  |  Link
cobo
Registered User
 
Join Date: Mar 2004
Posts: 118
Thanks. I could have sworn I was seeing a difference in the output of ColorYUV between using the values 0.5 and 1, but after checking carefully I see that it's correction is integer too.

Last edited by cobo; 26th September 2010 at 20:07.
cobo is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:23.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.