Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
14th January 2008, 21:51 | #1 | Link |
AviSynth plugger
Join Date: Nov 2003
Location: Russia
Posts: 2,183
|
ExInpaint plugin - Exemplar-Based Image Inpainting
Exemplar-Based Image Inpainting - removing large objects from images.
Selected objects has been replaced by a visually plausible background that mimics the appearance of the source region. Unlike other inpainting tools (logo removers), this filter does not blur image "holes". It tries to replace holes by similar texture elements (patches, exemplars, blocks) from frame image area, taking into account the image special points (with estimated confidence and priority). Block inpainting is started at hole border and propagates to its internal area, with isophote as a direction. Algorithm is based on article: Object Removal by Exemplar-Based Inpainting. A. Criminisi, P. Perez, K. Toyama. In Proc. Conf. Comp. Vision Pattern Rec., Madison, WI, Jun 2003. http://research.microsoft.com/vision...isi_cvpr03.pdf Code is based on project of Qiushuang Zhang http://www.cc.gatech.edu/grads/q/qsz...inpainting.htm First version of ExInpaint plugin is slow, not optimized, especially for large radius. It is spatial filter developed for still images. Only current frame is used. So, we can see temporal instability in this version. Potentially it may be used in video for: -film restoration (spots and scratches removal); -logo removal; -filling of empty areas (borders) after motion stabilization (compensation); -filling damaged areas after VHS capture; -frame increasing without resizing :-) It can be implemented differently, may be as a part of MVTools. Current version is mostly for experiment and testing. For logo removing and similar tasks you can also try XLogo, AVSInpaint, Greycstoration inpainting filters. |
19th January 2008, 06:26 | #2 | Link |
interlace this!
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
|
this is pretty amazing actually :|
i think i might hack this to do scratch concealment. [edit] seems to be RGB only? it works really well! try this: Code:
stackhorizontal(last.crop(0,0,64,last.height),last.blankclip(width=4,color=$0000ff),last.crop(68,0,652,last.height))#blankclip(width=652,color=$000000)) exinpaint(last,last,radius=16,xsize=3,ysize=3,color=$0000ff)
__________________
sucking the life out of your videos since 2004 Last edited by Mug Funky; 19th January 2008 at 06:34. |
23rd January 2008, 08:04 | #3 | Link |
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,259
|
little question:
Am I overlooking something? (can't find a avisynth filter, just a download for papers and a command line executeable) btw. tested the executeable with a little smaple I tool form google picture search: Original: http://www.postimage.org/image.php?v=aVV6qzA Masked: http://www.postimage.org/image.php?v=aVV6Ay9 Result: http://www.postimage.org/image.php?v=aVV6Fxr Cu Selur Last edited by Selur; 23rd January 2008 at 08:26. |
23rd January 2008, 08:33 | #5 | Link |
Registered User
Join Date: Oct 2001
Location: Germany
Posts: 7,259
|
thanks
http://avisynth.org.ru/ seems to be down atm, would be nice if someone who got the filter&co could up it to megaupload or something like it. -> works again Last edited by Selur; 23rd January 2008 at 12:54. |
24th January 2008, 02:49 | #6 | Link |
Registered User
Join Date: Nov 2005
Location: Wałbrzych, Poland
Posts: 105
|
@Fizick
Thanks for plugin. Two questions: 1. Does it make difference in speed which colorspace is used (RGB24, RGB32, YV12)? 2. Is there any chance in near future for multithreaded version of ExInpaint? It does not work (at least for me) with MT() nor with SetMTMode(). Maybe because it takes more than one clip as argument. |
24th January 2008, 16:59 | #7 | Link |
AviSynth plugger
Join Date: Nov 2003
Location: Russia
Posts: 2,183
|
1.1 You can measure it yourself
I almost finished next speed-optimized version (with SSE MMX). RGB32, RGB24, YV12 are ready. But YUY2 optimizing is not trivial task in MMX. 2.1. I do not develep m.t. filters (do not have hardware). 2.2. exinpaint can be used with some tricks with mask. 2.3. But according to algo the filter inpaints one block per a time (iteration).
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick I usually do not provide a technical support in private messages. |
13th April 2008, 18:48 | #8 | Link |
AviSynth plugger
Join Date: Nov 2003
Location: Russia
Posts: 2,183
|
I found that I developed version 0.2 at 26 January 2008
-even patch sizes instead of odd (with some approriate changes) -assembler MMX optimization of PatchTexture -use SAD instead of SSD -local max priority updating -option to estimate search radius automatically -added YUY2 format and YUV24 -added dilation of mask But I am afraid this version is not finally finished (beta or even alpha stage).
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick I usually do not provide a technical support in private messages. |
15th April 2008, 18:11 | #9 | Link |
Registered User
Join Date: Aug 2006
Posts: 77
|
Some sort of overlapping for block prevention would be nice too.
And some addition for the documentation: the color for yv12-mode is based on the binary representation of the mask color NOT the converted color of RGB. So $FFFFFF is quite colorful instead of white. Example: Code:
enlargedclip=clip.converttorgb().addborders(8,8,8,8,color=$FFFFFF).converttoyv12(matrix="pc.601") exinpaint(enlargedclip,enlargedclip,color=$FF8080) |
23rd April 2008, 17:20 | #10 | Link |
Registered User
Join Date: Apr 2008
Posts: 51
|
Thank you, it's very promising.
Hope the temporal instability can be fixed soon. btw, the border of inpaint area look sharper, somewhat noticeble if you look carefully, now if only we can blend the adjacent pixels a bit to make it more natural. |
18th September 2008, 20:17 | #11 | Link |
AviSynth plugger
Join Date: Nov 2003
Location: Russia
Posts: 2,183
|
Today I found that I forget to release version 0.2 beta in april (again!)
Prabably becourse of not many responces.... So, released now.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick I usually do not provide a technical support in private messages. |
19th September 2008, 00:07 | #12 | Link |
Resize Abuser
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
|
I've used it a couple of times on photos, thanks for the update!
__________________
Mine: KenBurnsEffect/ZoomBox CutFrames Helped: DissolveAGG ColorBalance LQ Animation Fixer |
3rd October 2008, 00:00 | #13 | Link |
Registered User
Join Date: Sep 2004
Location: Near LA, California, USA
Posts: 1,545
|
I'm trying to figure out how to remove a logo with this tool. I've yet to figure out how.
Any clues or tips would be appreciated. Here's a sample with a logo I'm trying to remove. http://www.sendspace.com/file/5atrwo
__________________
Pirate: Now how would you like to die? Would you like to have your head chopped off or be burned at the stake? Curly: Burned at the stake! Moe: Why? Curly: A hot steak is always better than a cold chop. |
3rd October 2008, 00:05 | #14 | Link |
Resize Abuser
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
|
U try this?
http://avisynth.org/mediawiki/Rm_logo ExInpaint is commented out, but you can uncomment it and try it out.
__________________
Mine: KenBurnsEffect/ZoomBox CutFrames Helped: DissolveAGG ColorBalance LQ Animation Fixer |
6th January 2011, 23:47 | #15 | Link | |
AviSynth plugger
Join Date: Nov 2003
Location: Russia
Posts: 2,183
|
Here is the message i have got recently from Jean (i have no clear answer to it):
Quote:
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick I usually do not provide a technical support in private messages. |
|
7th January 2011, 00:03 | #16 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
A black/white decision (either there is motion, or there is not) doesn't sound very promising. The forseeable result is false locking when the frame is slightly moving, with the funky garbling randomly kicking in when the holy motion threshold happens to be exceeded.
The better question is why ExInpaint delivers vastly different inpainting results when hardly anything changes between the frames ...
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
7th January 2011, 00:04 | #17 | Link |
Registered User
Join Date: Dec 2010
Posts: 2
|
Hey,
thanks for posting fizzick, that was me, with the mail, i couldn't post earlier here, because of the "you have to be 5 days registered to write"-rule. However, back to the problem/idea: I also talked with manao and kuukunen on IRC, it came out that what i wanted to do is most likely not possible. Because i simply can't take the result of the previous frame (at least i have no idea how, i tried with imagewrite and imageread, though that didnt work because the image is only read once, and not on every request) i tried with trim, duplicateframe.. So.. There's some additional magic needed i guess :-) Some people might say that this is useless at all, because it wouldn't give good results, however, the result's i've seen in my testing clip were good (especially if you just want to do that at borders. The less you want to add to the borders the better - This might be useful for turning 4/3 into 16/9 video. My idea was just to make masks around the borders, so that we have a left, top, bottom and right mask, and just checking whether there's motion in a mask - if there is, do inpainting, if not, use the previously inpainted one. Dunno. I weren't able to get something like that working and i was told it's impossible, so i gave up yet. Though if anyone else has any idea about it ... i'd be interested.
__________________
a computer without windows is like a chocolatecake without mustard |
7th January 2011, 00:30 | #18 | Link |
interlace this!
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
|
i think the algo would need to be rewritten almost completely - we'd be adding a new dimension to it.
the idea of using mvtools' vector field (and associated confidence levels, scene detection, etc etc) is pretty exciting though. the algorithm could track it's choices of exemplars (is this correct terminology?) along a contiguous scene, and update them locally when there's large enough changes, then possibly revert back if the scene returns to how it was earlier... i have no idea how to do this exinpaint already works beautifully for removing film scratches, but must be combined with a very heavy temporal blur (mv'd of course) to become invisible. also it can't handle very thick scratches without becoming apparent. i've saved the edges of quite a few films where the camera mag was unkind to them - those black vertical lines just vanish and leave no trace if you zero in well enough. i love it when clients think you've just performed an act of magic.
__________________
sucking the life out of your videos since 2004 |
22nd February 2013, 09:14 | #20 | Link |
Registered User
Join Date: Feb 2013
Posts: 1
|
exemplar based inpainting
I have problems to do about exemplar based inpainting..
I'm doing my final project and confused about some formula written on paper object removal exemplar based inpainting.. Could you help me? My problems are about to descript formula.. I'm confused about three formula: 1. Np... Is it just giving 1 to source region and. 0 to target region? 2. Dp = (np*(symbol of inverted triangle) Ip ) / alpha What is the mean of (symbol of inverted triangle) Ip. What should i do about that? 3. Psi Q... How to get that... |
Thread Tools | Search this Thread |
Display Modes | |
|
|