New Logo Removal Plugin

[AMSS0815]

Hi everybody,

finally I completed my logo removal plugin. This is the abstract from the help file:
Image Inpainting is the art of restoring destroied parts of an image by using information of valid parts of the image in a way, so that the human eye does not recognize the damaged areas (at least not at a first sight). In video processing image inpainting is often applied to videos in order to remove TV station logos. This plugin comes with the intention to provide a suit for the removal of logos, whether opaque or transparent. It provides algorithms for these tasks:

Logo detection: An obstacle is analyzed closely after being masked roughly by the user. The exact color map of the logo is determined as well as its alpha mask.
Deblending: Given the color image and alpha mask of an overlayed logo, the logo is removed where it is not opaque.
Logo inpainting: Opaque parts of the logo are interpolated. A fast method is used which has been published recently by Folkmar Bornemann and Tom März (see here (http://www-m3.ma.tum.de/twiki/bin/view/Allgemeines/Projekte#Fast_Image_Inpainting_Based_on_C)).

Finally, a function for calculating the distance function of a mask is provided, which can be used to expand or inflate masks as well as to provide smooth blending masks.


The logo detection and deblending are (more or less) what you (might) know from Karel Suhajda's DeLogo (http://neuron2.net/delogo132/delogo.html) filter for VirtualDub (but extended to YUV color spaces). The inpainting method is the implementation of a pretty new algorithm which produces much less blurring than the DeLogo filter. As I coded everything in pure C (without CPU optimizations) you shouldn't expect real-time performance:).

This mathematics and algorithms (concerning inpainting) behind this plugin are still under heavy development and the effects of the various parameters are not fully understood yet. If you have some experiences with this plugin and some ideas which settings produce good results in which situations, please post them here so the authors can learn from it. Please do not contact any of the authors (WB, FB, TM) directly. Also I should note that this algorithm has originally been developed for inpainting pictures, not movies, so you might observe strong discontinuities in time.

The plugin is attached to this thread as zip-file, including source code, a help file, and a demo script. Everything is licensed under the GNU GPL 2. See files AVSInpaint.htm and GNUGPLv2.txt for details. Have fun!

Regards,
AMSS

Edit: Added version 2008.02.23 as announced here. AMSS

[mitsubishi]

(but extended to YUV color spaces)

Sweet. I use the Delogo filter alot, but I only use the alpha deblending. There's maybe a better way of just applying an alpha mask, but that's how I do it.

With Delogo, I just find a blackest possible background the logo and the make my own masks in photoshop.

So I will test this out later.

[CruNcher]

Inpainting goes Avisynth nice work AMSS0815 :)
i think this here is interesting in those regards http://portal.acm.org/citation.cfm?id=1291237

it also seems a nice way for denoise/graining see http://www.mine.tku.edu.tw/advisors/tshih/tshih_projects/default.aspx Super-Resolution Inpainting
http://www.mine.tku.edu.tw/demo/inpainting/default.aspx

[Fizick]

will look to AVSInPaint!

Surpisingly, I prepare (almost finished, some doc need) two more avisynth impainting plugins :)
(different algos)
I partially interested in hole filling in motion stabilization and compensation, may be interpolation.

[Malcolm]

Hey, this is great!
I was just looking for an inpainting solution for avisynth and bang - the same day you release your plugin! :thanks:
Just your demo.avs - it's too much for me (it's really late now where i live) Could you just please provide a simple(!) realworld example. maybe a small clip; a logo file and a small avs?
I guess this would really help people to really understand what's needed to use your plugins!

Thanky a lot,
greetings,
Malcolm

[Fizick]

AMSS0815, why you do not use stdcall calling convention instead C? It is now default in avisynth, and extra avisynth_c.dll is not needed

Your plugin is not very slow! :)

GreyCstoration method in new GreyC plugin is probably slower:
http://forum.doom9.org/showthread.php?t=123081

[AMSS0815]

Hi there,
i think this here is interesting in those regards http://portal.acm.org/citation.cfm?id=1291237When I try to download the pdf-file, I'm forwarded to their homepage (http://portal.acm.org/portal.cfm). I think last summer I read some things about the stuff your other url points to: Seems they're both pretty good and pretty slow. However, I created AVSInpaint because I heard some talks about that algorithm. So I probably won't create more plugins. (Unfortunatelly I'm not in the image processing business anymore.)

Surpisingly, I prepare (almost finished, some doc need) two more avisynth impainting pluginsThe examples presented in the Microsoft-Paper (for ExInpaint) do look great! I have no opportunity to test it right now. The GREYCstoration seems to apply similar techniques as AVSInpaint (After skim reading their docs, I think they also compute something like Weickert's structuring tensor), but it seems their using an iterative scheme (which might reduce speed significantly)..?

I was just looking for an inpainting solution for avisynth and bang - the same day you release your plugin! ... Could you just please provide a simple(!) realworld example.Glad to help you. No - I avoid posting image or avi files. But Demo.avs really is a small example, only blown up with stuff that is not inpaint-related:Please ignore lines 1-130: They only provide the test clip with the logo and a framework to arrange the nice output clip.
Lines 133-139 actually create the test clip, so these lines would be replaced byMask4Analyze = ImageSource("LogoMask.png").ConvertToYV12
AVISource("Movie.avi").ConvertToRGB24
BigMovie=last
Clip=Crop(64,64,64,64) # Where the logo is and a lot space aroundLogoMask.png should be painted by hand, so it is white where the logo is (and some space around) and black elsewhere.
Lines 154-197 will find and remove the logo from Clip and put the result in RepairedClip.
Line 201 layeres the repaired (small) clip on top of the original clip (you will have to change the line by adding x=64 and y=64 or something else, and using BigMovie as base clip). Done.
There is also a small example in the help file (just put in your movie and try)...

why you do not use stdcall calling convention instead C
That's something I tried a lot about a year ago, but I failed. At the moment I'm using AviSynth_C.{dll|lib|h} in AviSynth_C-0.15.zip (http://kevin.atkinson.dhs.org/avisynth_c/avisynth_c-0.15.zip). When replacing AviSynth_C.{lib|h} by those in AviSynth_C-StdCall.zip (http://kevin.atkinson.dhs.org/avisynth_c/avisynth_c-stdcall.zip) (and recompiling and removing AviSynth_C.dll and the corresponding line in the avs-file), AviSynth 2.57 (build: Dec 31 2006) replies Unableto load C Plugin: AVSInpaint.dll. When replacing LoadCPlugin with LoadPlugin in the avs-file, I get LoadPlugin: unable to load "AVSInpaint.dll".:confused:
I'd appreciate any help in getting rid of this dll-file.

Your plugin is not very slow!:thanks: very much! Good to hear this.


And:thanks::thanks:for all the comments so far!!

Regards,
AMSS

[Fizick]

AMSS0815,
avisynth_c from Kevin has known typo bug with ACSC_USE_STDCALL instead of AVSC_USE_STDCALL.
Fixed version is distributed with Avisynth 2.5.7 setup:
avisynth_c.h is with FilterSDK option (and folder)
and avisynth.lib is with Extras option (and folder).
See also Yadif plugin package.

[AMSS0815]

Hi Fizick,Fixed version is distributed with Avisynth 2.5.7 setup:
avisynth_c.h is with FilterSDK option (and folder)
and avisynth.lib is with Extras option (and folder).I tried and failed. To be precise, I (re)placed AviSynth_C.h and AviSynth.lib with the ones comming with AviS 2.5.7, and changed the makefile (so it uses AviSynth.lib). It compiles fine, but linking with the lib-file results in a lot of errors like:AVSInpaint.obj(.text+0x6e1):AVSInpaint.c: undefined reference to `_imp__avs_release_video_frame@4'
AVSInpaint.obj(.text+0x1ec7):AVSInpaint.c: undefined reference to `_imp__avs_release_clip@4'
.....The line #define AVSC_USE_STDCALL 1 suggests that it uses standard calling convention (as far as I looked into the header file, this holds for the imported functions, too).

See also Yadif plugin package. I uncompressed it, and running make resulted in similar error messages (but not so much).
I wonder if my compiler is improperly configured :confused:.
By the way: It's gcc 4.2.1 (compiled with some older gcc), and linker is GNU ld 2.15.91 (delivered with MinGW).

AMSS

[Fizick]

my first attempt to build avsinpaint failled. - i replace only one avisynth_c.lib to avisynth.lib in make file and missed second entry :)

Error was similar to reported

But now avsinpaint (and yadif) build fine.

My config:
gcc version 4.0.4
from http://forum.doom9.org/showthread.php?t=108215

GNU ld version 2.17.50 20070129
Seems, it is important! get new binutils 2.17.50

I do not use mingw directly, but with minsys or with Codeblock.

[Reuf Toc]

I've wrote a function for AVSInpaint. I hope it will help peoples who have difficulties to use this sympathic plugin. Since it's my first function I ask for leniency. :p

#____________________________________________________________________________________
# _____ _ _ ______
# |_ _| (_) | | | ____|
# | | _ __ _ __ __ _ _ _ __ | |_| |__ _ _ _ __ ___
# | | | '_ \| '_ \ / _` | | '_ \| __| __| | | | '_ \ / __|
# _| |_| | | | |_) | (_| | | | | | |_| | | |_| | | | | (__
# |_____|_| |_| .__/ \__,_|_|_| |_|\__|_| \__,_|_| |_|\___|
# | |
# |_|
#_____________________________________________________________________________________



# InpaintFunc is a new delogo function using AVSInpaint.dll an AviSynth C plugin developed by AMSS.
#See thread : http://forum.doom9.org/showthread.php?t=133682 to download this plugin and get
#more information about it. Just an advertise : AVSInpaint plugin is rather slow so be patient ;-)


#_______________________________________________________________________________________
#
# History :
#_______________________________________________________________________________________


# Actual revision : InpaintFunc v1.13

# Author : Reuf Toc

# Requirement : AVSInpaint.dll


# Changes :

#Rev 1.00 : First release 2008/01/30
#Rev 1.01 : Add debug for "mode". Removed two useless operations.
#Rev 1.02 : Added "Speed" parameter.
#Rev 1.03 : Fixed bug in PAR calculation. Fixed bug with non-RGB32 mask.
#Rev 1.10 : Fixed bug due to overlay. Added post-processing + a lot of things.
#Rev 1.11 : Added Show parameter and InpaintAssist function. Process in RGB24 instead of RGB32.
#Rev 1.12 : Fixed bug in "loc" calculation
#Rev 1.13 : Fixed bug for .ebmp creation (did not happen in several cases)




Find last version of InpaintFunc on the wiki : InpaintFunc (http://avisynth.org/mediawiki/InpaintFunc)

[geminihc]

sorry if this questions very stupid, but i'm using FFDshow with avisynth. what exactly do i type in avisynth section in ffdshow to use this..?

[foxyshadis]

It'd doubtful that you could do this in realtime even if you put it in ffdshow. You'd also have to change the parameters for every movie's logos.

[Adub]

@ Reuf Toc

I have added your function to the Avisynth wiki.
Here:
http://avisynth.org/mediawiki/InpaintFunc

Feel free to edit and update your function on there when ever you feel like it. (Plus, it makes the function easier to find and maintain.)

[Reuf Toc]

Function updated (here and on the Wiki)

The two majors changes are a correction of a bug caused by overlay and the addition of a basic post-processing feature.

Changelog :

Fixed bug due to overlay()
Added basic post-processing.
Suppression of Cropval parameter.
Check if the logo analysis has already been computed.
Default value are differents.
Added parameters to tweak InpaintLogo.
Changed debug variables from int to bool.
Changed mask creation method.

[AMSS0815]

Hi everybody,

I updated AVSInpaint to version 2008.02.23. Main change is the removal of AviSynth_C.dll. Thanks to Fizick, this file isn't necessary anymore (Plugin is still loaded with LoadCPlugin). Further (minor) changes are listed in the help file. The file is attached to the original post of this thread.

Nice work, Reuf Toc. I really have to admitt, I didn't spend to much time on making this thing user friendly.

Have fun,
AMSS

[Poutnik]

I have a question to InPaincFunc about manual setting of crop area to localize logo ( loc "x1,y1,-x2,-y2" ).

Should I set it to logo quite tightly (with some border) to speed up the things ?
Or, is it a trade off between speed if tight and quality if large enough to have more out-of-logo info ?r
Or it does not matter and "TL" is enough ?

[Reuf Toc]

I don't know exactly how AVSInpaint works internally but from what I understood, the number of pixels computed to restore the damaged area is defined by "radius" parameter.

So the minimum border around the logo should be at least equal to the radius. If this condition is fulfilled, setting an larger area does not increase quality but only time computation.

So I recommend to use your own loc parameter, especially if your source is HD

[pbristow]

Hi! I'm trying to use AVSinpaint via the InPaintFunc script, as follows:

<code>
LoadCPlugin ("C:\Program Files\AviSynth 2.5\C_plugins\AVSInPaint.dll")

AVISource("GP1.TESTCLIP.avi")
Trim(0,10)

InpaintFunc(mask="G:\GP\Logo_1280x720.bmp", loc="48,48,152,52", speed=20, mode="both", show=false)
</code>


AVSInPaint keeps failing with the error "mask is full". Can anyone explain what that message means? I've searched the docs provided, and the forums, and tried Googling for ' AVSInPaint "mask is full" ' - No luck. (Except for a thread on a Chinese forum, where, if I'm correctly interpreting the machine translation, they don't know what it means either!)

N.B. I'm using the older version of AVSInPaint because I couldn't persuade the newer one to load. (Tried every combination I could think of, using loadplugin/loadCplugin, putting the .dll file in plugins/c_plugins folder, etc. - again, no joy.) If this is a known prob that's fixed in a newer version, could someone point me to it and clarify how it has to be loaded?

Thanks.

[Reuf Toc]

Hi! I'm trying to use AVSinpaint via the InPaintFunc script, as follows:

<code>
LoadCPlugin ("C:\Program Files\AviSynth 2.5\C_plugins\AVSInPaint.dll")

AVISource("GP1.TESTCLIP.avi")
Trim(0,10)

InpaintFunc(mask="G:\GP\Logo_1280x720.bmp", loc="48,48,152,52", speed=20, mode="both", show=false)
</code>


AVSInPaint keeps failing with the error "mask is full". Can anyone explain what that message means? I've searched the docs provided, and the forums, and tried Googling for ' AVSInPaint "mask is full" ' - No luck. (Except for a thread on a Chinese forum, where, if I'm correctly interpreting the machine translation, they don't know what it means either!)


"mask is full" means that your mask is "full white" after cropping. Either you cropped too much the clip with the "loc" values you used or there is a problem with your mask (make sure that black parts of your mask are really black).
If you post a screenshot of your source and your mask, I can take a look.


N.B. I'm using the older version of AVSInPaint because I couldn't persuade the newer one to load. (Tried every combination I could think of, using loadplugin/loadCplugin, putting the .dll file in plugins/c_plugins folder, etc. - again, no joy.) If this is a known prob that's fixed in a newer version, could someone point me to it and clarify how it has to be loaded?

Thanks.
Have you tried Load_Stdcall_Plugin, the alias for LoadCPlugin ? AVSInpaint is still a C plugin so loadplugin can't work.

[pbristow]

<quote>"mask is full" means that your mask is "full white" after cropping. </quote>

Ah! I understand now. :)

From looking at the source, I could see that it should only say "Mask full" if the mask was 100% white, but my mask isn't (I thought)... but I was overlooking the "after cropping" detail. :) For my first experiments, I'm using a simple white rectangle against full black as my mask, and in my keenness for speed, I see that with my loc="" string I've cropped well inside the white rectangle.

<quote>Have you tried Load_Stdcall_PluginM</quote>
Not yet. I'll try the newer version with that later.

Thanks for your help. :)

[StainlessS]

@Reuf Toc

Love your script, tried the newer alternative (dont off hand remember name)
but prefer yours. Can I though suggest that with your next script update, you include
the below mod:

catch (dummy)
{
show=true
in2.analyzelogo(Masque).trim(0,-1).converttoRGB32
imagewriter(ID,0,1,"ebmp")
}""")

Namely the "Show=true" line.

makes it much more usable and would obviate the need for me
to replace it when you do make an update.

Forces the script to actually save the bitmaps, dont really
know how it's supposed to work without it.

Also, can I suggest that you and others viewing this thread
see update just posted minutes ago here:

http://forum.doom9.org/showthread.php?t=154559

Have had great success using my script as a post processor
to AvsInpaint, InpaintFunc combo.

thanx, be good :)

[StainlessS]

@Reuf Toc

See this post for future problem in InPaintFunc.avs

http://forum.doom9.org/showthread.php?p=1479855#post1479855

[Beak]

Hello. This is my first try at logo removal. I am marginally familiar with scripting but have never seen the error I am getting.

I have AVSInpaint.dll in my Avisynth 2.5 plugins folder. The script is as follows

import("C:\Program Files (x86)\Winnydows\XviD4PSP5\dlls\AviSynth\functions\AudioFunctions.avs")
import("C:\Program Files (x86)\Winnydows\XviD4PSP5\dlls\AviSynth\functions\VideoFunctions.avs")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\AVSInpaint.dll")
Import("C:\Program Files (x86)\AviSynth 2.5\lsfmod\InpaintFunc.avs")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\FFT3dGPU.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\MT.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\MCTemporal Cleaner\mt_masktools-25.dll")

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\MCTemporal Cleaner\RemoveGrainSSE3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\MCTemporal Cleaner\RepairSSE3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\lsfmod\RSharpen.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\lsfmod\WarpSharp.dll")

Import("C:\Program Files (x86)\AviSynth 2.5\lsfmod\LSFMod.avs")
DirectShowSource("G:\In the heat of the night\2011_4_24_15_58_17.avi", fps=23.976, convertfps=true)

ConvertToYV12()
Trim(1168,159461)


MT("""InpaintFunc(last, mask="G:\In the heat of the night\Logo White.bmp", loc="br", AR=16.0/9.0, mode="both", speed=10, ppmode=3, pp=75)""", threads=4, overlap=8)

I get an error for line 3 that says "AVSInpaint.dll is not an Avisynth 2.5 plugin" Does this plugin only work for 2.6? Is there a 2.5 version available.

My LSFMod scripts work etc. I just hadn't added the parameters for LSFMod yet.

I would really appreciate any help that you can offer. Thanks BK

[Gavino]

I get an error for line 3 that says "AVSInpaint.dll is not an Avisynth 2.5 plugin" Does this plugin only work for 2.6? Is there a 2.5 version available.
AVSInpaint.dll is not a standard plugin (it is writtten in C, not C++) and must be loaded by
LoadCPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\AVSInpaint.dll")

[Beak]

Thank You. The script now loads with the following error

Mask dimension does not match that of input file.

I checked and they are both 1920X1080.

Any hints for this?