Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
26th December 2017, 11:07 | #1 | Link |
Guest
Posts: n/a
|
1px Horizontal line removal (at varying vertical locations)
I'm currently working on an encode of a tool-assisted speedrun and I've come to noticed a horizontal line in certain situations in-game. It's likely an emulation bug, but due to the fickle nature of syncing input playback, options that could potentially fix it cannot be changed. This is also the reason behind the excessive (and stretched) resolution, which is easy enough to fix once the line is removed.
Throughout the game, the output goes to and from 4:3 to letterboxed widescreen. It cycles to this letterbox resolution by having black bars fall down from the top and raise up from the bottom over a series of frames (the exact amount varies with the emulator). The line itself only appears when these black bars do, 24 pixels above the top of the gameplay window. This means as the black bars fall in, the line falls in, too. Furthermore, UI elements are drawn on top of the line. If the game is displaying in 4:3, the line is not present at all. Here is a short clip with several frames exhibiting the problem in it's various states (selected at random). Because I need the footage to maintain as much accuracy as possible (despite it's alarming inaccuracy), I decided the best course of action regarding the line's removal would be to first detect if the line was present, and if so, replace the entire line with an average of the lines directly above and below it. This would ensure that the UI elements weren't distorted (and if they were, it would be nearly impossible to tell). I initially was unaware of the fact the line fell in with the black bars and thought it stayed permanently at y=385 and so I used the following script to detect and remove the line: Code:
# Load the source AVISource("zelda_test.avi") # Create the reference clip for the check # Horizontal values set to avoid UI elements, vertical values set based on the line's location with a 2-pixel buffer LineReferenceTop = Last.Crop(900,2,-2500,-2498) LineReferenceBottom = Last.Crop(900,387,-2500,-2474) LineReference = StackVertical(LineReferenceTop, LineReferenceBottom) # Create the output clip for when a line is detected # OutputLine averages the pixel above and below the line in order to keep UI elements intact OutputTop = last.crop(0,0,0,-2496) OutputLine = Average(last.crop(0,383,0,-2496), 0.5, last.crop(0,385,0,-2494), 0.5) OutputBottom = last.crop(0,385,0,0) LineFixed = StackVertical(OutputTop, OutputLine, OutputBottom) # Perform a check comparing the test clip to a completely black clip # If the result is 0, test area is black and line should be replaced ConditionalFilter(LineReference, last, LineFixed, "RGBDifference(LineReference.BlankClip())", ">", "0", false) Last edited by thecoreyburton; 26th December 2017 at 11:15. |
27th December 2017, 12:44 | #3 | Link |
Guest
Posts: n/a
|
If it makes you feel better, neither did I.
I'm thinking that if perhaps the clip has padding added to the top, a detection could be performed looking for n black pixels, followed by a row of non black pixels, followed by n black pixels. If the detection is true, then whatever y value the line was detected at could have the fix applied. Unfortunately, I'm not sure how to go about this in terms of AVISynth script though. Last edited by thecoreyburton; 28th December 2017 at 12:43. |
30th December 2017, 14:03 | #4 | Link |
Guest
Posts: n/a
|
I did a lot of experimentation and spoke to a few people and came up with the following script:
Code:
AVISource("zelda_test.avi") global v1 = false global v2 = false global v3 = false LineIndex = DetectLine(32, 96, 960, 2240, 4860, 480, 20) (LineIndex == 0) ? Last : Last.FixLine(LineIndex) function FixLine(clip c, int i) { # Function by TheCoreyBurton c_top = c.crop(0,0,0,-(c.height-i+1)) c_bottom = c.crop(0,i,0,0) c_top_blend = c.crop(0,i-2,0,-(c.height-i+1)) c_bottom_blend = c.crop(0,i,0,1) c_line = Average(c_top_blend, 0.5, c_bottom_blend, 0.5) output = StackVertical(c_top, c_line, c_bottom) return output } function DetectLine(clip c, int block_width, int x1, int x2, int x3, int x4, int ymax, int buffer) { # Function by TheCoreyBurton (props to feos, Warepire and creaothcean) return_index = 0 c1 = c.crop(x1,0,block_width,0) c2 = c.crop(x2,0,block_width,0) c3 = c.crop(x3,0,block_width,0) c4 = c.crop(x4,0,block_width,0) c_stack = StackHorizontal(c1, c2, c3, c4).AddBorders(0,buffer,0,0) GScript(""" for (index = buffer+1, ymax+buffer+1, 1) { global p1 = c_stack.crop(0,index-1-buffer,0,buffer) global p2 = c_stack.crop(0,index-1,0,1) global p3 = c_stack.crop(0,index,0,buffer) global d1 = 0 global d2 = 0 global d3 = 0 #FrameEvaluate(p1, "global d1 = RGBDifference(p1, p1.blankclip)") #FrameEvaluate(p2, "global d2 = RGBDifference(p2, p2.blankclip)") #FrameEvaluate(p3, "global d3 = RGBDifference(p3, p3.blankclip)") #v1 = (d1 == 0) ? true : false #v2 = (d2 == 0) ? false : true #v3 = (d3 == 0) ? true : false return_index = v1 ? v2 ? v3 ? index-buffer-1 : return_index : return_index : return_index } """) return return_index } Last The first problem I'm having is that I need to somehow store the results of RGBDifference from each detection into values (and I have no idea how to do that). I've commented out the six lines that didn't work as intended and I only really need a true/false response to make it work, but my current lines of code were not functioning correctly. The script should return "0" if no line was found, or if a line was found it should return the index so the FixLine function can be carried out. The second problem that may arise (depending on how the first is dealt with) is that this needs to be evaluated on a per-frame basis, as the line's position is dynamic and changes regularly. I also attempted an alternate solution to the problem (a "cheat"). I decided to use a variation of the original ConditionalFilter line I wrote in conjunction with global variables in the following manner: Code:
AVISource("zelda_test.avi") global v1 = false global v2 = false global v3 = false LineIndex = DetectLine(32, 96, 960, 2240, 4860, 480, 20) (LineIndex == 0) ? Last : Last.FixLine(LineIndex) function FixLine(clip c, int i) { # Function by TheCoreyBurton c_top = c.crop(0,0,0,-(c.height-i+1)) c_bottom = c.crop(0,i,0,0) c_top_blend = c.crop(0,i-2,0,-(c.height-i+1)) c_bottom_blend = c.crop(0,i,0,1) c_line = Average(c_top_blend, 0.5, c_bottom_blend, 0.5) output = StackVertical(c_top, c_line, c_bottom) return output } function DetectLine(clip c, int block_width, int x1, int x2, int x3, int x4, int ymax, int buffer) { # Function by TheCoreyBurton (props to feos, Warepire and creaothcean) return_index = 0 c1 = c.crop(x1,0,block_width,0) c2 = c.crop(x2,0,block_width,0) c3 = c.crop(x3,0,block_width,0) c4 = c.crop(x4,0,block_width,0) c_stack = StackHorizontal(c1, c2, c3, c4).AddBorders(0,buffer,0,0) GScript(""" for (index = buffer+1, ymax+buffer+1, 1) { global p1 = c_stack.crop(0,index-1-buffer,0,buffer) global p2 = c_stack.crop(0,index-1,0,1) global p3 = c_stack.crop(0,index,0,buffer) ConditionalFilter(p1, p1.v1black, p1.v1else, "RGBDifference(p1, p1.BlankClip())", ">", "0", false) ConditionalFilter(p2, p2.v2black, p2.v2else, "RGBDifference(p2, p2.BlankClip())", ">", "0", false) ConditionalFilter(p3, p3.v3black, p3.v3else, "RGBDifference(p3, p3.BlankClip())", ">", "0", false) return_index = v1 ? v2 ? v3 ? index-buffer-1 : return_index : return_index : return_index } """) return return_index } function v1black(clip c) { global v1 = true return c } function v1else(clip c) { global v1 = false return c } function v2black(clip c) { global v2 = false return c } function v2else(clip c) { global v2 = true return c } function v3black(clip c) { global v3 = true return c } function v3else(clip c) { global v3 = false return c } Last It's also worth noting that due to the nature of the project, the code in DetectLine for selecting the index may be incorrect. I've been able to give it user-defined numbers so far, but I won't be able to see if it stops at the correct point in the loop until the loop itself is working. Any testing should probably be done with this in mind. Last edited by thecoreyburton; 30th December 2017 at 14:21. |
30th December 2017, 17:05 | #5 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
On sample frame 2, there are hearts and some ball type things intruding into the black border, can the left and or right edges of the borders be guaranteed
free of such hearts and baubles ? EDIT: For detection of border present.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
30th December 2017, 18:02 | #6 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
How does this look to you ? (simple preliminary test)
Code:
ORG=AVISource("D:\zelda_test.avi") ORG TEST_LEFT_W=320 TEST_RIGHT_W=320 TEST_H=420 LFT=Crop(0,0,TEST_LEFT_W,TEST_H) RGT=Crop(Width-TEST_RIGHT_W,0,0,TEST_H) StackHorizontal(LFT,RGT) Function TestIt(clip c) { c Bingo = RT_RgbInRangeLocate(RLo=0,GLo=0,Blo=0,RHi=0,GHi=0,BHi=0,Baffle_W=Width,Baffle_H=2) Bingo ? RT_Subtitle("%d %d %d %d",RGBIRL_X,RGBIRL_Y,RGBIRL_W,RGBIRL_H) : NOP Bingo ? Overlay(Last,Last.BlankClip(Width=Width/2,Height=1,Color=$FF0000),x=Width/4,y=RGBIRL_H-24) : NOP Return Last } Return ScriptClip("Return Testit") EDIT: Frame 0 (Top LHS and top RHS corners only, stacked horizontal) Red marks the spot. Numbers are the top border coords when located, red mark relative bottom of border (-24). Correctly ID's line on each of the sample frames suffering from the artifact. EDIT: Frame 8
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 24th December 2019 at 10:40. |
31st December 2017, 02:49 | #7 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
This any good ?
Code:
GSCript(""" Function FixLine(clip c, int i, Bool Fix, Bool Show) { c if(i>=0) { if(FIX) { if(i==0) { StackVertical(crop(0,1,0,1),crop(0,1,0,0)) } else { m=StackVertical(crop(0,i-1,0,1),crop(0,i+1,0,1)).BilinearResize(Width,3).crop(0,1,0,1) StackVertical(crop(0,0,0,i),m,crop(0,i+1,0,0)) } } Show ? Overlay(Last.BlankClip(Width=Width*7/8,Height=3,Color=$FFFF00),x=Width/16,y=i-1) : NOP Show ? Subtitle(String(current_frame,"%.0f") + String(i,"] Line @ Y=%.0f"),Size=Height/32.0) : NOP } Else { Show ? Subtitle(String(current_frame,"%.0f] Border NOT FOUND"),Size=Height/32.0) : NOP } return Last } """) AVISource("D:\zelda_test.avi") #AVISource("D:\zelda_test2.avi") TEST_LEFT_W=320 TEST_RIGHT_W=320 TEST_H=408 FIX =true # Fix Artifact if True SHOW=true # Mark line with height of 3 (make more visible) in Yellow. DC=StackHorizontal(Crop(0,0,TEST_LEFT_W,TEST_H),Crop(Width-TEST_RIGHT_W,0,0,TEST_H)) SSS=""" i = DC.RT_RgbInRangeLocate(RLo=0,GLo=0,Blo=0,RHi=0,GHi=0,BHi=0,Baffle_W=DC.Width,Baffle_H=23) ? RGBIRL_H-24 : -1 FixLine(Last,i,FIX||!SHOW,SHOW) return Last """ Return ScriptClip(SSS) #.BicubicResize(854,480) EDIT: Changed Baffle_H from 2 to 23. EDIT: Added FIX. EDIT: Mod TEST_H=408
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 24th December 2019 at 10:40. |
31st December 2017, 03:14 | #8 | Link | |
Guest
Posts: n/a
|
Quote:
Thanks for all the replies, StainlessS. That last script is fantastic. I tried it on a few sections of the full 400,000 frame source and only came across one situation where it didn't function as intended. Here's a new sample, have a look at frames 3 and 4. It's not failing to detect the line, but it rather looks like it's detecting inconsistencies below the line. |
|
31st December 2017, 03:21 | #9 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Can you retry with current script (I changed Baffle_H to 23 while you were replying).
EDIT: Quote:
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 31st December 2017 at 03:29. |
|
31st December 2017, 03:30 | #10 | Link |
Guest
Posts: n/a
|
No worries, I really appreciate the help!
That fixed the issue on frame 4 of the new sample (Where the line was detected significantly lower than it's actual coordinate), but not frame 3 of the new sample (where the line is detected only a single pixel lower). |
31st December 2017, 03:32 | #11 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
OK, thanks. I'll down the current sample.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
31st December 2017, 03:54 | #12 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Change to TEST_H=408 and retest please.
EDIT: Can you answer this too. Quote:
EDIT: Erroneous detection due to top line of that frame 3 being totally black, ie looks like border. TEST_H=408, is presumably maximum height of top border.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 31st December 2017 at 04:03. |
|
31st December 2017, 04:13 | #13 | Link |
Guest
Posts: n/a
|
That seems to have fixed the problem. In a few days I'll be able to apply this to the entire clip and see the results, but I'm optimistic. Thank you.
Regarding the way the video was acquired, the graphics plugin is notorious for sometimes adding garbage data to the border of the output. I've only ever seen this amount to a single pixel in width or height (depending on which border it's adding to) and would usually deal with it by using Crop(1,1,-1,-1). |
31st December 2017, 04:16 | #14 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Thank you, good luck and let us know how it goes.
EDIT: May fail if top border less than max 408, and some of top lines of non border is near black (right across the width). EDIT: I'm trying to improve border detection where top line of non border is near black. (Probably require RT_Stats v2.0 beta)
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 31st December 2017 at 16:39. |
1st January 2018, 11:27 | #16 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
OK, wanna give this a whirl, it dont need RT_ v2.0
Code:
GSCript(""" # Allocate Format strings Globally only once, not for each frame. Global G_String_0 = "%s" Global G_String_1 = "%d] Border NOT FOUND" Global G_String_2 = "%d] Line=%d Border Detect FAIL : Border NOT Full scan Width (X=%d:W=%d, should be 1:%d)" Global G_String_3 = "%d] Line=%d Border Detect FAIL : Border NOT Start @ top of frame : (Top=%d, should be 1)" Global G_String_4 = "%d] Line=%d Bad Detect (pop=%.2f%%) : Check frame" Global G_String_5 = " Re-Detected OK @ Line=%d (pop=%.2f%%)" Global G_String_6 = " Upward scan did not find artifact : LineBest=%d popBest=%.2f%% " Global G_String_7 = "%d] Line = %d (pop=%.2f%%)" Function FixLine(clip c,clip DC, Int BordHi,Float BordPerc, Int MarkLo,Float MarkPerc,Int Matrix, \ Bool Fix,Bool Fix_ReDetect,Bool Subs, Bool Mark,String LogFile,Bool VerbLog) { c frm=current_frame locWid = DC.Width - 2 # Exclude left and right edges (crud on sample frame 8, LHS) Line=DC.RT_YInRangeLocate(x=1,y=1,w=locWid,Baffle_W=(locWid+1)/2,Baffle_H=23,Lo=0,Hi=BordHi,Matrix=Matrix, \ Thresh=BordPerc) ? YIRL_Y+YIRL_H-24 : -1 if(Line<0) { S=RT_String(G_String_1,frm) VerbLog ? RT_WriteFile(LogFile,G_String_0,s,Append=True) : NOP Subs ? Subtitle(S,Size=Height/32.0) : NOP } else if(YIRL_W!=locWid){ S=RT_String(G_String_2,frm,Line,YIRL_X,YIRL_W,locWid) RT_WriteFile(LogFile,G_String_0,s,Append=True) Subs ? Subtitle(S,Size=Height/32.0) : NOP } else if(YIRL_Y!=1 && Line!=0){ # We also excluded top scanline in border detect (crud) S=RT_String(G_String_3,frm,Line,YIRL_Y) RT_WriteFile(LogFile,G_String_0,s,Append=True) Subs ? Subtitle(S,Size=Height/32.0) : NOP } else { vi = DC.RT_YInRange(x=1,y=Line,w=locWid,h=1,Lo=MarkLo,Hi=255) * 100.0 # retest, test for lighter line if(vi <= MarkPerc) { # Percent of light pixels too low, scan upwards, try re-detect RT_WriteFile(LogFile,G_String_4,frm,Line,vi,Append=True) Linechk = -1 vbest=vi jbest=Line for(j=Line-1, 0, -1) { # Line, Fixed in BLUE v = DC.RT_YInRange(x=1,y=j,w=locWid,h=1,Lo=MarkLo,Hi=255) * 100.0 if(v>vbest) { jbest=j vbest=v } if(v > MarkPerc) { Linechk = j RT_WriteFile(LogFile,G_String_5,Linechk,v,Append=True) Fix = Fix && Fix_ReDetect vi = v j = -1 # Break with j = -2 } } if(Linechk<0) { RT_WriteFile(LogFile,G_String_6,jbest,vbest,Append=True) Fix=False } Line=Linechk } if(Line>=0) { if(Fix) { if(Line==0) { StackVertical(crop(0,1,0,1),crop(0,1,0,0)) } else { m=StackVertical(crop(0,Line-1,0,1),crop(0,Line+1,0,1)).BilinearResize(Width,3).crop(0,1,0,1) StackVertical(crop(0,0,0,Line),m,crop(0,Line+1,0,0)) } } Mark?Layer(Last.BlankClip(Width=Width*7/8,Height=3,Color=$FFFF00).ResetMask,"add",257,Width/16,Line-1):NOP S=RT_String(G_String_7,frm, Line, vi) VerbLog ? RT_WriteFile(LogFile,G_String_0,s,Append=True) : NOP Subs ? Subtitle(S,Size=Height/32.0) : NOP } } return Last } """) AVISource("D:\zelda_test.avi" , "D:\zelda_test2.avi") TEST_LEFT_W=320 TEST_RIGHT_W=320 TEST_H=408 DC=StackHorizontal(Crop(0,0,TEST_LEFT_W,TEST_H),Crop(Width-TEST_RIGHT_W,0,0,TEST_H)) ################### BORDHI = 3 # Is possible border scanline if pixel population 0 -> BORDHI is Greater than BORDPERC. BORDPERC = 5.0 # (BORDPERC may work as low as 0.0, Dont go too high, max maybe 10.0 [Baffle is main detector]) ### MARKLO = 4 # Is artifact line if pixel population MARKLO -> 255 is Greater than MARKPERC. MARKPERC = 5.0 # ### Matrix = 3 # 2 = PC.601 : 3 = PC.709 (RGB -> Luma Conversion for detection) FIX = false # Fix Artifact if True FIX_REDETECT = false # Fix Bad detections that were re-detected OK. SUBS = True # Subtitles in Yellow (may OMEM if not SMALL=True). MARK = True # Mark line with height of 3 (make more visible) in Yellow. SMALL = True # MARK/FIX/SUBS detect clip only (fails on big clip for Subtitle, Out Of Memory) LogFile = "MyLog.Log" # Log filename VERBLOG = true # More Verbose Logging ################### RT_FileDelete(LogFile) VC = SMALL ? DC : Last Return VC.ScriptClip("Return FixLine(Last,DC,BORDHI,BORDPERC,MARKLO,MARKPERC,Matrix, \ FIX,FIX_REDETECT,SUBS,MARK,LogFile,VERBLOG)") #.BicubicResize(854,480) have excluded left, right and top single pixels from detection area, should still work as before, no need to crop crud. If problems, I may have to use RT_ v2.0 as it has separate threshold for border detection, for h and v. EDIT: If you need post further samples, then only post DC clip, a helluva lot smaller (and minus audio).
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 17th January 2018 at 16:00. |
1st January 2018, 23:42 | #17 | Link |
Registered User
Join Date: Jul 2010
Location: Germany
Posts: 357
|
It might result in faster processing if you just put black lines there:
Code:
# Crop(1, 1, -1, -1).AddBorders(1, 1, 1, 1) RemoveGarbage function RemoveGarbage(clip c) { c h = BlankClip(c, 1, Width, 1, color=$FF000000) v = BlankClip(c, 1, 1, Height, color=$FF000000) c .\ Layer(h, x= 0, y= 0).\ Layer(h, x= 0, y=Height-1).\ Layer(v, x= 0, y= 0).\ Layer(v, x=Width-1, y= 0) } Last edited by creaothceann; 1st January 2018 at 23:50. |
2nd January 2018, 02:16 | #19 | Link |
Registered User
Join Date: Jul 2010
Location: Germany
Posts: 357
|
No, just assumed that cropping and adding borders involves copying lots of memory around.
EDIT: in a quick test with this code and VirtualDub's "preview output from start" function, using the function resulted in a very slight speedup (35 vs. 38 seconds). Code:
a = Version.ConvertToRGB32.Trim(0, -1) b = BlankClip(a, 400000) Layer(b, a) # Crop(1, 1, -1, -1).AddBorders(1, 1, 1, 1) RemoveGarbage |
2nd January 2018, 02:34 | #20 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Perhaps letterbox a better option to blacken borders:- http://avisynth.nl/index.php/Letterbox
Docs (EDIT: Its one of those built-in filters that is very easy to forget about) Quote:
EDIT: The detector should still be able to detect and correct bad scanline @ line 0, even though the border detect starts @ scanline 1. The bad scanline is bottom of border - 24 pixels [EDIT: total border height, incl top scanline, - 24 pixels, ie upwards].
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 2nd January 2018 at 18:38. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|