Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th August 2013, 11:08   #21  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi martin53!

Your script good work for me with small adjustments. One problem it is speed on my 4 core 2 thereaded CPU (8 virtual core) I see only speed near 8-10 fps for SD (720x576). Main reason is dither plugin or Stainless plugin? How can increase speed using MT. I find work version only with SetMTMode(5) before call Your script. May be can use avstp? How?
yup.
yup is offline   Reply With Quote
Old 30th August 2013, 18:34   #22  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 404
yup,
thank you that you use the script!
Quote:
Originally Posted by yup View Post
How can increase speed
You are absolutely welcome to help finding ways. The version I published yesterday is again optimized for function, not for speed. I will optimize a few things, e.g. parallelise the statistics with StainlessS' plugins and remove steps that can be avoided. Currently, the script needs up to three runtime environments; One might be obsolete and I will check how far the other two can be combined.
It was already a great help that the MYStats plugin was made and extended after I had asked for it, and I do of course hope that I can get more help of that kind in the future.

But especially when it comes to MT, I was out of luck every time I tried until now. You and everyone else are invited to understand the script - I commented every step with some short statements - and contribute speed optimizations.
martin53 is offline   Reply With Quote
Old 30th August 2013, 20:44   #23  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 8,864
Further to PM, What is RGBBalancedShift ?

I'm guessin that Gavino could sort the prob, bUt I cannot

The big G is such a wonderful individual.
.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 9th October 2013 at 20:09.
StainlessS is offline   Reply With Quote
Old 31st August 2013, 02:13   #24  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 8,864
OK. I tell a lot of porkies,
BUT, I think Gavino would be the first person to say that He is the one for predictions.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 9th October 2013 at 20:09.
StainlessS is offline   Reply With Quote
Old 1st September 2013, 13:09   #25  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 404
Quote:
Originally Posted by StainlessS View Post
Further to PM, What is RGBBalancedShift ?
It is in the package in 1st post. In short, it does a RGBAdjust with normalized factors so the luma stays same.

In full it is more complicated to avoid clipping, be able to deal with YUV & RGB, be called from in-/outside RTE, reduce color banding ...

Gavino responded to the bug thread in the developer forum. Very kind!
martin53 is offline   Reply With Quote
Old 1st September 2013, 17:31   #26  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,341
Why do you turn these scripts not into a plugin?
Wilbert is offline   Reply With Quote
Old 1st September 2013, 20:43   #27  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 404
Quote:
Why do you turn these scripts not into a plugin?
for two reasons.

* I like prototyping in script language, and all users have a full chance to see what happens inside - and maybe improve it.

* I avoided to set up a plugin development environment till now. I am not unexperienced in C programming, but not accommodated to C++. And from reading some developer threads, I am a bit scared.

I also have the ulterior motive that one of the experienced plugin developers can't refuse to try it.

But to warn you: One part of the script is kinda 'academic' and easy to transer to C. But the fuzzy logic to deal with many real world illuminations is more a sort of to play with and gather experience than copy and compile. Let me just mention three situations I still struggle with:
- Frames showing sky and/or water only. These are blue but need to be distinguished from a scenery that suffers from a too high color temperature
- Closeups of face/body, vegetation, rock/gravel. These may be very biased towards different colours and should not be greyed out
- Clips that were taken at tungsten light typically still show white as the brightest colour because very bright orange can't be coded into the clip, or the sensor is saturated. It is neccessary to overdo the correction only for this situation and to the appropriate amount.

The three basic approaches I cited at thread start do not cope with such challenges. The script modifies 'Shades of Gray' with some additional rules and modified parameters, with an adaptive colour-local approach etc.
martin53 is offline   Reply With Quote
Old 1st September 2013, 22:34   #28  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 8,864
Dont be a scaredy cat M53.

Hope you've seen this thread in devs (Avisynth CPP Interface for C Programmers):
http://forum.doom9.org/showthread.php?t=163082

Once you've got the standard interface worked out, you can more of less forget CPP and just code C.

(Is easier than the C interface).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 1st October 2013, 14:02   #29  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi martin53!
I try Your last script and find small bug .
1. Your define variable show after first use.
reduce = Default(reduce, show?1:3) # reduction factor for the stats clips
only after 6 lines I see
show = Default(show, false)
2. I get error at line
s = replace(replace(s, "--", "+"), "+-", "-")
I install Grunt, GScript, Mystats, RT_Stats.
Please advice.
yup.
yup is offline   Reply With Quote
Old 1st October 2013, 20:13   #30  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 404
yup,
Quote:
Originally Posted by yup View Post
reduce = Default(reduce, show?1:3) # reduction Please advice.
Please change to
Code:
reduce = 1
I learned that it is better not to scale down the analysis frame, in order to leave the RGB relations of single bright pixels unchanged.

I also did a lot more of research, but was still not satisfied with the latest results, so I took a break and did not publish more recent test versions.
The hardest problem with my test clips is that with outdoor scenes, Max-RGB (also called white patch) is the best algorithm. But with indoor scenes, cameras over-expose lamps and show them as full white, while the scene itself is too warm. So, a means to detect this fundamental defect of the scene is needed - in order to switch between algorithms accordingly - and I could not yet find useful and reliable clip properties for that.
martin53 is offline   Reply With Quote
Old 2nd October 2013, 04:09   #31  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
martin53
Quote:
Originally Posted by yup View Post
2. I get error at line 691
s = replace(replace(s, "--", "+"), "+-", "-")
I install Grunt, GScript, Mystats, RT_Stats.
Please advice.
yup.
At Your script exist code starting from line 689
Code:
	s = "((x<128+"+D_POS+" ? (y<128+"+D_POS+" ? 1 : (0.43*((128-x-0.37+"+D_POS+")^1.08) + 0.38*(exp((128-x-0.37+"+D_POS+")*0.08)-1))/(y-128-"+D_POS+")-1) : ( y>127+"+D_POS+" ? 0 : 1-(x-128+0.18-"+D_POS+")^0.8/(128-y+"+D_POS+")) > 0) & (x<128+"+D_NEG+" ? (y<128+"+D_NEG+" ? 0 : 1-(0.43*((128-x-0.37+"+D_NEG+")^1.08) + 0.38*(exp((128-x-0.37+"+D_NEG+")*0.08)-1))/(y-128-"+D_NEG+")) : ( y>127+"+D_NEG+" ? 1 : (x-128+0.18-"+D_NEG+")^0.8/(128-y+"+D_NEG+")-1) > 0) & y-x<="+minTempUV+" & y-x>="+maxTempUV+") ? 255 : 0"
	s = replace(replace(s, "--", "+"), "+-", "-")
	s = mt_polish(s)
When i load script in VirtualDub I get error related to replace function. Which plugin have replace function?
yup.
yup is offline   Reply With Quote
Old 2nd October 2013, 18:04   #32  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 404
Oops, I didn't notice this lacks in the script, it is part of my standard function library.
Replace() is a shortcut for StrReplace:
Code:
Function StrReplace(string s,string find,string replace) # Repeated, string replacements
# Original:- http://forum.doom9.org/showthread.php?t=147846&highlight=gscript           By Vampiredom, Gavino, IanB
#{i=s.FindStr(find)return(i==0?s: s.LeftStr(i-1)+replace+s.MidStr(Strlen(find)+i).StrReplace(find,replace))}
# Converted to use RT_Stats RT_StrAddStr() to avoid 2.58/2.6a3 string concatenation bug:-
{i=s.FindStr(find) return(i==0?s:RT_StrAddStr(s.LeftStr(i-1),RT_StrAddStr(replace,s.MidStr(Strlen(find)+i)).StrReplace(find,replace)))}
#=====================================================================================================================
Function Replace(string s,string find,string replace) { StrReplace(s, find, replace) }
martin53 is offline   Reply With Quote
Old 2nd October 2013, 22:08   #33  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 8,864
m53, suggest convert to use rt_strreplace, is plugin, faster & no un-freeable temp strings created.

EDIT: Also can replace

Code:
s = replace(replace(s, "--", "+"), "+-", "-")
with

Code:
s = RT_StrReplaceMulti(s, "--"+Chr(10)+"+-" ,  "+"+Chr(10)+"-" )
but the Chr(10) separated multiline strings might best be constructed outside of ScriptClip/GScript rather than at each iteration.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 9th October 2013 at 20:00.
StainlessS is offline   Reply With Quote
Old 3rd October 2013, 07:27   #34  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi martin53!
After introduce show before first call and add replace function I get
Subtitle at frame
Code:
ScriptClip: Function did not return a video clip! (Was an int)
my script simple
Code:
AVISource("selbuild.avi")
s=AssumeTFF().Crop(16,12,-16,-12).ConvertToYV12(interlaced=true).SeparateFields().SelectEven()
AWB(s)
StackVertical(last,s)
Please explain.
yup.
yup is offline   Reply With Quote
Old 9th October 2013, 16:11   #35  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,994
Got the same error after making those small corrections needed to open the script. I'd like to give the function a shot on some VHS stuff I need to process
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 9th October 2013, 22:06   #36  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 8,864
Quote:
Originally Posted by yup View Post
After introduce show before first call and add replace function I get
Subtitle at frame
Code:
ScriptClip: Function did not return a video clip! (Was an int)
Check error reported in DebugView(Google) during catch(err).

@m53,

Suggestions:-

Drop MYStats, convert all to RT_Stats latest funcs.

Wrap Default(floatarg,floatnum) ---> Float(Default(floatarg,floatnum))
# If user supplied function arg as int var (Gavino Tip, avoid unintended int calcs).

Convert eg cR.MYPNorm() ---> RT_RgbChanStats(flgs=req_funcs,chan=-1)
# where same funcs called for R,G,and B channels, avoid conv to Y8 and multiple individual channel individual func calls.
# Guess that you are already doing that.

Convert eg Subtitle("#") ---> RT_Subtitle("#")
# faster but perhaps not necessary nor as pretty

Also see prev post.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 12th October 2013, 09:25   #37  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,994
DebugView shows this: [2896] RT_Debug: ScriptClip 3 ######## MYPlaneMin: Mask clip dissimilar dimensions.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 12th October 2013, 16:06   #38  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 8,864
M53, is there any reason you chose height >= 600 as rec709 switch as opposed to eg height > 576 ?

It is better than using RT_Stats default Width>720, shall change RT to same as yours.

EDIT:
The fault reported by Yup and Boulder, resides here:-
Code:
      cu = cYUV.UToY
      cv = cYUV.VToY

      Umin = MYPlaneMin(cu, cYUV, MaskMin=UVRANGE_CHECK_YMIN, MaskMax=255)
The debugview message "MYPlaneMin: Mask clip dissimilar dimensions." is issued by MYPlaneMin, (Throws the error)
cu and cv are quarter size (w/2,h/2) of mask when YV12.
If YV24 supplied to func then does not throw error but returns YV12 instead.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 13th October 2013 at 00:02.
StainlessS is offline   Reply With Quote
Old 13th October 2013, 18:34   #39  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 404
First, my apologies to all I did not answer earlier.
StainlessS, thanks for the suggestion. I copied the 600 threshold for the matrix somewhere, don't remember where. The future version on my HD is already mxRec = clp.width<=720?"Rec601":"Rec709". I agree that it is of advantage to be in line.

Luckily, I just read your post, because I was on the way to implement MYStats instead of MRGBStats because I hadn't seen that RT_Stats now includes all functions I need. Thanks you assembled everything together.

As for the bugs with the published script - it is difficult to keep up at the moment, because several things happened in parallel: the xxxStats functions grew better and were merged, I debugged the script and made it more robust, and I improved the algorithm. I can impossibly publish todays working version - it would cause too many questions and errors.

The most annoying and challenging issue is that consumer cameras depict warm white lamps as RGB=fully white, although the scene itself is balanced towards yellow. I.e. the white is heavily biased and unsuitable for AWB purposes. This has heavy impact because in almost every situation, the 'max RGB' alias 'White Patch' algorithm is the one which meets expectations best. In the meantime, I prepared three sets of reference clips with 'difficult' footage - and alas, the algorithm still produces inacceptable output with some scenes.

My proposal is that I'll concentrate on switching over to RT_Stats and provide a version that runs, and then continue with the improvements of the algorithm. Please have a few days more patience! Thanks!

EDIT: RT_Stats issue, see RT_Stats thread

Last edited by martin53; 13th October 2013 at 19:19. Reason: RT_Stats issue
martin53 is offline   Reply With Quote
Old 13th October 2013, 19:56   #40  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 404
Quote:
Originally Posted by StainlessS View Post
M53, is there any reason you chose height >= 600 as rec709 switch as opposed to eg height > 576 ?
Firesledge (aka Cretindesalpes) in dither.html:
Quote:
When the parameter is not defined, ITU-R BT.601 and ITU-R BT.709 are automatically selected depending on the clip definition (considered as HD if ≥ 600 lines).

For me, width>720 is a reasonable distinguishing mark for HD. I paid homage to Cretindesalpes' greater experience.
martin53 is offline   Reply With Quote
Reply

Tags
awb, color, colorbalance

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.