apply copied color to another place in the image

[chmars]

Hello,

Film transfer.
I have white perforations, they disturb during dark scenes.
I don't wish to crop the image too much, so part of the perfos will stay.
I Borrowed from Zorr Delta Restore Script.
I managed to:
-make a mask,
-blur it a vertically more than horizontally,
-fill the perfos with dark gray,


SetMemoryMax(8000)
threads = 8
SetFilterMTMode("DEFAULT_MT_MODE", 2)

source1=FFVideoSource(464-13_63.mov").KillAudio()
source1 = Crop(source1, 500,0,-130,-60)
perfo = LumaDeltaMask(source1, ">", brightness=1.15, spotSize=90)
fill_color = BlankClip(source1,color=$26282d)

expandtimes = 7
for (i = 1, expandtimes) {
perfo = perfo.mt_expand( chroma="-127")
}

perfo = perfo.FastBlur(2, 5)

filled = Overlay(source1,fill_color,mask=perfo)

result=StackHorizontal(source1, filled)

return(result)


#########################################################################################################################

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, string direction, float "brightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)

gray = BlankClip(source, color=$c0c0c0)
# diff
diff = Grayscale(mt_makediff(source, gray ))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

return mask
}
Prefetch(threads)


https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/1NuaNCeMUE7aSg2/download

Source video (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/XAqwhj9gw8j6CFo/download)

Even cropped more, it's not discrete, because the tint is wrong.

I found some filters to analyse the color of a pixel but I didn't manage to use them with with my color format.
I also don't figure out how to say "take an average of the color at a place, relative to the mask place".

I'd like to analyse the color near the perforations and use it to fill them. This, before grading, on a YUV422P16 image.

Any idea of directions to search for the right tools?

[AVIL]

Hi, you can try avsinpaint;

http://avisynth.nl/index.php/AvsInpaint

As an idea, not tested personally.

[chmars]

Thanks, going to check it right away.

[chmars]

I have the feeling AvsInpaint will be a bit heavy, also to find all right parameters.
Before to go forward with it, I tried something, which seem to work.
Almost sure it's not the optimal way, though...

Click to see the video. Before grading-rotating-cropping stage
https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/JsDUXzPvOUrcY2z/download (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/h1JBtcsLnTryYuH)






source1=FFVideoSource("some_file.avi").KillAudio()

#~ mypixel = Crop(source1,400,320,-1850,-900)
#~ return(mypixel)

ScriptClip(source1, """
source1 = last
mypixel = Crop(source1,400,320,-1850,-900)

s1cnv = convertbits(mypixel, 8)
s1cnv = ConverttoYV16(s1cnv)
mypixel = Converttorgb(s1cnv, matrix = "rec601")

MyR = int(AverageR(mypixel))
MyG = int(AverageG(mypixel))
MyB = int(AverageB(mypixel))

bgdColor = MyR*65536 + MyG*256 + MyB

perfo = LumaDeltaMask(source1, ">", brightness=1.17, spotSize=90)

fill_color = BlankClip(source1,color=bgdColor).addgrain(var = 25, uvar = 0).FastBlur(1.3)#(1,0,0,0,False,true)


---------------------------------------------------
expandtimes = 9
for (i = 1, expandtimes) {
perfo = perfo.mt_expand( chroma="-127")
}

perfo = perfo.FastBlur(3, 5)


filled = Overlay(source1,fill_color,mask=perfo)
source1.height/4)

#~ return(Subtitle(filled, String(bgdColor)))
return(filled)

#########################################################################################################################
#Borrowed from Zorr Delta Restore. Thanks to him.
function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots ----------------------------------------------------
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-127")
return mask
}

function LumaDeltaMask(clip source, string direction, float "brightness", int "spotSize") {
# direction is ">" if you want mask to contain areas where source is brighter than filtered by "brightness", direction "<" selects where source is darker
# adjust brightness to change how bright objects are selected, typical values between 1.0 and 1.1, larger value = brighter object
# spotSize controls the smallest spot size allowed in the mask, typical values between 0 and 3

brightness = Default(brightness, 0.9)
spotSize = Default(spotSize, 2)

gray = BlankClip(source, color=$c0c0c0)
# diff
diff = Grayscale(mt_makediff(source, gray ))

# mask
mask = mt_lut(diff, expr="x range_half "+String(brightness)+" * " + direction + " 255 scalef 0 ?")

# remove too small spots
mask = RemoveSmallSpots(mask, spotSize)

return mask
}
""")



Lesson:
:eek::) It's very ugly to see these big orange variations, even after cropping. Less is more. Gonna stay with white, or black for dark scenes.

[real.finder]

did you tried crop then AvsInpaint?

[johnmeyer]

I deal with 8mm sprockets all the time, since my Workprinter exposes the entire frame. You usually have a MUCH bigger problem than the one you are addressing: exposure when doing the transfer. Those two white-hot sprocket holes completely screw up the exposure. Therefore, I use tape and two pieces of cardboard in the optical path to the camera to mask off the two holes. It makes a huge difference because pure black is pretty close to the normal black levels in film whereas pure white from those sprocket holes is many stops brighter than the brightest film scene.

The next thing is to decide what to do with the holes. There is no solution that works on all scenes, but you can use your NLE to create a duplicate of the frame, shift it left or up (for the upper left sprocket hole), blur it a bit and, once shifted, composite a "hole-sized" section of the adjacent part of the frame into the hole, feathering the edges. You can sometimes get better results if you flip the little segment you are using so that the edges match perfectly. Vegas makes this pretty easy to do. I can't speak for other NLEs. If you are a perfectionist, you may need to do this differently for different scenes. For instance, if the scene is outdoors and to top is sky, you want to shift horizontally, and the result can be almost perfect.

[chmars]

did you tried crop then AvsInpaint?

Thakns, no. I confess: I saw the page, I ran away. Never used Avsinpaint before.
Long time to tweak, or?
I like to "see the mechanic" so I wrote(/copied) this script. But you both might be right, I should take my courage and try.

You usually have a MUCH bigger problem than the one you are addressing:
..Those two white-hot sprocket holes completely screw up the exposure. Therefore, I use tape and two pieces of cardboard in the optical path...

I set the aperture (i mean fstop) as the sprockets to be almost on top of the histogram. I had this problem when I was using another lens but not anymore. (almost not, only in very low/unexposed scenes: I feel it if I have to push the blacks).


The next thing is to decide what to do with the holes. There is no solution that works on all scenes, but you can use your NLE to create a duplicate of the frame, shift it left or up (for the upper left sprocket hole), blur it a bit and, once shifted, composite a "hole-sized" section of the adjacent part of the frame into the hole, feathering the edges. You can sometimes get better results if you flip the little segment you are using so that the edges match perfectly. Vegas makes this pretty easy to do. I can't speak for other NLEs. If you are a perfectionist, you may need to do this differently for different scenes. For instance, if the scene is outdoors and to top is sky, you want to shift horizontally, and the result can be almost perfect.
Yes, like do some deshaker? Thanks for the trick. But this work is already time consuming a lot!
One more trick to automate in Avisynth with options and flavours :)
Can you afford to do this often? Also grading these shifted crazy footages takes ages, isn't it? (but the whole process is pleasant + addictive)

[johnmeyer]

Deshaker has nothing to do with the sprocket holes, although perhaps you are referring to what happens to the sprocket holes when you stabilize the footage. As always, it is important to do things in the right order. Also, in my version of the script, I added a temporary crop that lets you completely eliminate all residual black borders before stabilizing, but then add back those borders in order to get back to the original dimensions, but without resizing, which you always want to avoid.

[chmars]

...although perhaps you are referring to what happens to the sprocket holes when you stabilize the footage...
I was making reference to some trick for some deshaker that used a part of mirrored slice of the image to fill the missing parts/borders.