Log in

View Full Version : Inpainting; use former and later frames?


nji
24th October 2023, 22:01
Recently I came to repair and delogo an older movie.
The regions to repair are static but also dynamic (spatial).
To do my job I would like to not only inpaint the dynamic areas
by encounting the spatial environment
(that's what InpaintDeLogo (https://forum.doom9.org/showthread.php?t=176860)filter does quite well),
but also the temporal environment.
As being dynamic a "logo" reveals the things behind earlier or later.
If I get it right that's something InPaintDeLogo can't, right?
How could I do this, with AviSynth or any other tool?
Thanks for hints!

poisondeathray
24th October 2023, 22:37
You can use a "temporal inpainter", such as propainter (or dozens of other temporal inpainters, but propainter is probably one of the better ones right now . Not perfect, but substantially better than the previous generation of temporal inpainters) . The main requirement is you need usable data in other frames for it (or any temporal inpainter) to work properly; or you have to make that data "usable". e.g. if you have a film with other defects , let say dirt - it could copy those defects in to the patch area. You'd either have to prefilter the dirt, or mask out the dirt as well.

It's difficult to use if you're not used to python projects , and there is no vpy or avs port yet. It's also limited by GPU memory. More memory means you can use more reference frames, better settings. There is no CPU mode, and I tried modifying it but no success

See this post for an example, more info, and a partial list of other temporal inpainters . It's not an exhaustive list, there are dozens more.
http://forum.doom9.org/showthread.php?p=1992324#post1992324

Other useful tasks/scenarios - filling in sproket holes , fix ratty film edges, stabilization edge fill , expand field of view (like photo stitching, but for video) , generate clean plates, object removal (e.g. remove ex-wife from a shot :devil:) etc... I might post some other examples later . Very useful. I'd love to see a VPY / AVS port

spoRv
25th October 2023, 20:08
When I was looking for a frame interpolation tool, I found this: https://github.com/n00mkrad/flowframes
It uses Python, and a Windows GUI very easy to use even for someone like me that does know nothing about Python!

I wonder if it's possible to do something similar for ProPainter - yes, and AVS port would be ideal, but a "simple" Windows GUI will be more than appreciated!!!