Calculating row information?

[HappyLee]

Hi. I'm working on a script to automatically remove specific broken lines from broken frames (kind of like DeScratch, but more precise).

Right now, the only thing troubling me is that, I can't find any filter or function to calculate row information. I want to set a row completely white if its average luma is above a certain value, or otherwise, completely black.

For the last script, I used something like this:
StackHorizontal(last,last,last,last).BilinearResize(4,height)
But I don't find it precise nor smart.

Technically, I can do something like this:
ConvertToY8()
StackVertical(Crop(0,0,0,1).ScriptClip("..."),Crop(0,1,0,1).ScriptClip("..."),Crop(0,2,0,1).ScriptClip("..."), ...)
But I don't find it elegant enough, because the video has 480 rows, not 4 or 8. Even if I achieve it with functions & recursion, I'm not sure if the speed would be OK, since StackVertical would be used 479 times.

So I'm humbly asking advanced video workers here, if there's any filter that can apply to rows, or calculate row information. Thank you. :)

[StainlessS]

Perhaps this might push you in the right direction, (req avs2.6)


Colorbars.Trim(0,-100).ConvertToY8.KillAudio
H=Height
SeparateRows(H)

# WeaveRows(H).Info

Return Last


Converts 100 frames @ 640x480 , to (100*480) frames @ 640x1

[StainlessS]

OK, here tis, seems working OK, and really quite nippy too.


Colorbars.Trim(0,-100).ConvertToY8.KillAudio
ORG=Last
H=Height
SeparateRows(H)
KR = Last.BlankClip(Length=1,Color_YUV=$008080)
WR = KR.Invert

LIMITHI = 100.0 # <= is Target (100.0 for test on Colorbars greyscale)
SHOW = True
DEBUG = False

CondS="""
ave=AverageLuma # ave of current frame (row)
ret = (ave<=LIMITHI) ? 1 : 0 # 0 = Not detect Black(White Row return), 1=Detect Black(Black Row return)
(DEBUG) ? RT_DebugF("%d] Ave=%f ret=%d",current_frame,ave,ret) : NOP
return ret
"""

ConditionalSelect(Last,CondS,WR,KR) # 0=WR : 1=KR
WeaveRows(H)

(SHOW) ? StackHorizontal(ORG,Last) : Last
Return Last

https://i.postimg.cc/CKnrRGwc/row-00.jpg

EDIT: You might need use latest test version Avs+, think there were possible problems specifying Color_YUV for Y8.

EDIT: I'm gettin' 22FPS on x86 and 27FPS on x64 Core Duo quad (using single core).

EDIT: Colorbars Greyscale, Ave of upper block is 110.298439, of middle block 74.504684, of bottom block 72.639061.

EDIT: Mod with 2 limits, gonna set black for middle bar


LIMITLO = 74.0 # >= is Target # We're gonna Black middle block
LIMITHI = 75.0 # <= is Target # Ditto
SHOW = True
DEBUG = False
###############
Colorbars.Trim(0,-100).ConvertToY8.KillAudio # Or your source filter
###############
ORG=Last
H=Height
SeparateRows(H)
KR = Last.BlankClip(Length=1,Color_YUV=$008080)
WR = KR.Invert

CondS="""
ave=AverageLuma # ave of current frame (row)
ret = (LIMITLO <= ave <= LIMITHI) ? 1 : 0 # 0 = Not detect Black(White Row return), 1=Detect Black(Black Row return)
(DEBUG) ? RT_DebugF("%d] Ave=%f ret=%d",current_frame,ave,ret) : NOP
return ret
"""

ConditionalSelect(Last,CondS,WR,KR) # 0=WR : 1=KR
WeaveRows(H)

(SHOW) ? StackHorizontal(ORG,Last) : Last
Return Last

https://i.postimg.cc/HkTfLYsV/row-01.jpg

[HappyLee]

Great idea, StainlessS. I've never thought of using SeparateRows() before. Thank you so much!

Here's my solution:


ConvertToY8()
H=Height
SeparateRows(H)
ScriptClip("""mt_lut(AverageLuma()>80? "255":"0")""")
WeaveRows(H)

[StainlessS]

Great stuff, you got an AvsMeter FPS for that, and also comparison for my version [on real test clip] ?

EDIT: Maybe should create your Black/White row Mt_lut's external to scriptclip, you're creating them at every frame[EDIT: frame/row].
ALso, I did a mod version above, maybe of use for something else.

[StainlessS]

For my original script, I got about 22FPS.
For your script I got about 12FPS with colorbars.
With below, I'm gettin' about 30FPS.

Colorbars.Trim(0,-100).ConvertToY8.KillAudio
H=Height
SeparateRows(H)
KR=Last.mt_lut("0")
WR=Last.mt_lut("255")
CondS = """
return (AverageLuma >100) ? WR : KR
"""
ScriptClip(CondS)
WeaveRows(H)


EDIT: and this is really a little bit back to front, "return (AverageLuma >100) ? WR : KR"
I would prefer "return (AverageLuma<=100.0) ? KR : WR" as we want to detect bad rows, not good rows, but dont really matter.

[real.finder]

more speed

Colorbars.Trim(0,-100).ConvertToY8.KillAudio
H=Height
SeparateRows(H)
KR=Last.BlankClip(color_yuv=color_black)
WR=Last.BlankClip(color_yuv=color_white )
CondS = """
return (AverageLuma >100) ? WR : KR
"""
ScriptClip(CondS)
WeaveRows(H)

[HappyLee]

You guys are great. I thought mt_lut was fast, but somehow BlankClip is faster. :)

[real.finder]

even faster

Colorbars.Trim(0,-100).ConvertToY8.KillAudio
H=Height
SeparateRows(H)
KR=Last.BlankClip(color_yuv=color_black)
WR=Last.BlankClip(color_yuv=color_white )
gConditionalFilter(last,WR, KR,"AverageLuma >100")
WeaveRows(H)

[StainlessS]

R.F,
Inventive use of RGB colors red channel to set Y of Y8. :)

[real.finder]

R.F,
Inventive use of RGB colors red channel to set Y of Y8. :)

what about ExtractR() ? or ShowRed("Y8") in avs 2.6

[wonkey_monkey]

Using y8_rpn (https://forum.doom9.org/showthread.php?t=172601) and avoiding yucky runtime filters:


# source filter here, making source implicit "last" variable (y8_rpn is weird about implicit last)

source = last

last.y8_rpn("
0 B x 1 < ? @B^
[c0] B + @B^
B w /
")

average = crop(last.width-1,0,0,0)

stackhorizontal(average, source.converttoy8)

last.converttoyv24.y8_rpn("
0 255 [c0] B x 1 < ? @B 128 < ?
")


The "128" in the second call to y8_rpn is the threshold.

Bear in mind y8_rpn is very old, and 32-bit only. expr() may be able to replace it but I'm not sure how its variables work between pixels/rows.

[StainlessS]

Mod, supports both Row & Column mask, with LimitLo and Limithi Range select. (avs 2.6 req)

EDIT: DONT USE THIS, SEE NEXT POST


# EDIT: DONT USE THIS, SEE NEXT POST
ROW = True
LIMITLO = 74.0 # >= is Target
LIMITHI = 100.0 # <= is Target
INCOLOR = $008080 # Set where in target range
OUTCOLOR = $FF8080 # Not in target range
SHOW = false # Return StackHorizontal, original as Y8, and mask.
###############
Colorbars.Trim(0,-100).convertToY8
MskByRowAveY(Row=ROW,LimitLo=LIMITLO,LimitHi=LIMITHI,InColor=INCOLOR,OutColor=OUTCOLOR,Show=SHOW)
Return Last

Function MskByRowAveY(clip c, Bool "Row", Float "LimitLo", Float "LimitHi", Int "InColor", Int "OutColor", Bool "Show") {
# Where AveLuma of pixel Row/Coloumn is between LimitLo<===>LimitHi, then set to Incolor, else OutColor. Colors Specified as YUV, where only Y8 returned.
c myName="MskByRowAveY: "
Row=Default(Row,true) LimitLo=Default(LimitLo, 0.0) LimitHi=Default(LimitHi,127.5)
InColor =Default(InColor ,$000000) OutColor=Default(OutColor,$FF8080) Show=Default(Show,False)
Assert(0.0 <= LimitLo <= LimitHi,myName+String(LimitLo,"0.0 <= LimitLo(%f)") + String(LimitHi," <= LimitHi(%f)"))
Assert(LimitHi <= 255.0,myName+String(LimitHi,"LimitHi(%f) <= 255.0"))
ConvertToY8.KillAudio O=Last
(Row) ? SeparateRows(O.Height) : SeparateColumns(O.Width)
InC = Last.BlankClip(Color_YUV= InColor) OutC= Last.BlankClip(Color_YUV=OutColor)
Last.ConditionalFilter(InC,OutC,String(LimitLo,"(%f<=AverageLuma<=")+String(LimitHi,"%f)"))
(Row) ? WeaveRows(O.Height) : WeaveColumns(O.width)
Return (SHOW) ? StackHorizontal(O,Last) : Last
}


Original
https://i.postimg.cc/68t5NZ5Y/row2-02.jpg

With Row=True
https://i.postimg.cc/Z0Hwctvn/row2-00.jpg

With Row=False
https://i.postimg.cc/2LyhGtgL/row2-01.jpg

EDIT:
Note to Pinterf, ConditionalFilter Fails where uses Length=1 in BlankClip, dont think it should.
InC = Last.BlankClip(Length=1,Color_YUV= InColor) OutC= Last.BlankClip(Length=1,Color_YUV=OutColor)

[StainlessS]

Fixed a couple of weird problems in previous version.


ROW = True
LIMITLO = 74.0 # >= is Target
LIMITHI = 100.0 # <= is Target
INCOLOR = $008080 # Set where in target range
OUTCOLOR = $FF8080 # Not in target range
SHOW = True # Return StackHorizontal, original as Y8, and mask.
MATRIX = "rec601" # For ConvertToY8, Match to LimitLo & LimitHi
###############
Colorbars.Trim(0,-100)

MskByRowAveY(Row=ROW,LimitLo=LIMITLO,LimitHi=LIMITHI,InColor=INCOLOR,OutColor=OUTCOLOR,Show=SHOW,matrix=MATRIX)
Return Last

Function MskByRowAveY(clip c, Bool "Row", Float "LimitLo", Float "LimitHi", Int "InColor", Int "OutColor", Bool "Show",String "Matrix") { # http://forum.doom9.org/showthread.php?p=1886301#post1886301
# Where AveLuma of pixel Row/Coloumn is between LimitLo<===>LimitHi, then set to Incolor, else OutColor. Colors Specified as YUV, where only Y8 returned.
# Requires:- Either <Avs v2.60 with Grunt plugin>, OR <Avs+>.
Function FuncNameExists(String Fn){Try{Eval(Fn+"()")B=True}catch(e){Assert(e.FindStr("syntax")==0,"FuncNameExists: Error in Function Name '"+Fn+"'")B=(e.FindStr("no function named")==0)}Return B}
Function IsAvsPlus() { Return FuncNameExists("autoloadplugins") }
Function HasGrunt() { Return FuncNameExists("GSCriptClip") }
c myName="MskByRowAveY: "
Row=Default(Row,true) LimitLo=Default(LimitLo, 0.0) LimitHi=Default(LimitHi,127.5)
InColor =Default(InColor ,$008080) OutColor=Default(OutColor,$FF8080) Show=Default(Show,False) Matrix=Default(Matrix,"rec601")
Assert(2.6 <= VersionNumber,myName+"Avs v2.60+ required for Separate/Weave/Rows)")
Assert(IsAvsPlus || HasGrunt,myName+"Either Avs+ OR Grunt required)")
Assert(0.0 <= LimitLo <= LimitHi <= 255.0,myName+String(LimitLo,"0.0 <= LimitLo(%f)") + String(LimitHi," <= LimitHi(%f) <= 255.0"))
ConvertToY8(matrix=Matrix).KillAudio O=Last
(Row) ? SeparateRows(O.Height) : SeparateColumns(O.Width)
InC = Last.BlankClip(Color_YUV= InColor) OutC= Last.BlankClip(Color_YUV=OutColor)
Last.ConditionalFilter(InC,OutC,String(LimitLo,"(%f<=AverageLuma<=")+String(LimitHi,"%f)"))
(Row) ? WeaveRows(O.Height) : WeaveColumns(O.width)
Return (SHOW) ? StackHorizontal(O,Last) : Last
}

Minor EDITs (and added Matrix arg).

Original
https://i.postimg.cc/68t5NZ5Y/row2-02.jpg

With Row=True
https://i.postimg.cc/Z0Hwctvn/row2-00.jpg

With Row=False
https://i.postimg.cc/2LyhGtgL/row2-01.jpg

[Cary Knoop]

Cool! Is something like this doable in Vapoursynth as well?

[StainlessS]

I imagine so, but not by me [we dont play together as yet].

EDIT: Using JohnMeyer Parade clip, frame 1000, orig, Row=true,Row=False. [ LIMITLO = 74.0 LIMITHI = 100.0 Matrix="rec601" ]
https://i.postimg.cc/ykGrxkcw/sub-00.jpg (https://postimg.cc/ykGrxkcw)

EDIT: You can also use the Zebra Avisynth plugin for something a bit similar. [Maybe I should have persuaded that to assist].

https://i.postimg.cc/bZBkkx9B/CMAD-zpsae30b66c.jpg (https://postimg.cc/bZBkkx9B)

https://i.postimg.cc/K4HRBgFN/Equid.jpg (https://postimg.cc/K4HRBgFN)

EDIT: Added this functionality to Zebra():- http://forum.doom9.org/showthread.php?t=167663

[StainlessS]

Just thought I'de point out that I added Matrix arg to post #14 a few days back. [for RGB -> Y8 conversion, affects LimitLo and LimitHi functionality]