Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th January 2010, 01:45   #1  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,327
Fusion - pyramidal image processing for video [experimental]

Even more updated!

The latest version (a rewrite which only implements fuse and fusemask) is at http://horman.net/fusion.zip



____________________________________________________________________________________________________________

Updated: see here

This is Fusion, a set (if you can call "two" a set) of filters that use image pyramids to blend clips together (more commonly used in the creation of HDR images and image stitching).

I'm not sure it's all that useful for video, but I've written the library now and AviSynth seemed like an interesting way to play with it. As opposed to a normal mix of clips with a mask, the filter fuse(clip1,clip2,mask) will "decompose" frames with a high pass filter, making a stack of different levels of detail. These levels are then blended separately before being recomposed into a final image.

A picture paints a thousand words, so:



On the left is a "naive" blend with a widely feathered mask. Notice that the stalk is semi-transparent, and the feather line is a little textureless. With multi-level blending, on the right (as output by fuse) textures are preserved and the blending of underlying colour is much wider, yet still preserves features such as the stalk. It fudges up the background - not sure what that's about but I suspect it's just way it works.

A possibly more useful application than creating hybrid fruit is in boosting the levels of underexposed video:



The mask is a threshold of a blurred copy of the brightened image (top right), and fuse uses this to blend the two different exposures without (or at least, with very little) haloing.

The two filters in the current .dll are:

Code:
# NB: all input clips must be RGB32

fuse(clip1, clip2, mask)
# mask should be black/white only, unless extra blending of fine details is required
# in the mask, black represents clip1, white represents clip2

undersharpen(image, level, amount)
# this will sharpen/blur only one level of an image
# level=0 is the top level, i.e. finest detail
# useful values for amount range from -0.33 to +1.00
undersharpen doesn't seem to do anything very useful, but I didn't know that when I wrote it so I've left it in.

I'm not yet sure what other applications image fusion may have with video, so if you know of any please let me know and I'll try to implement them. There might (or might not) be good reasons to try extending this to include the temporal dimension...

David

Last edited by davidhorman; 24th November 2014 at 19:15.
davidhorman is online now   Reply With Quote
Old 16th January 2010, 23:05   #2  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,327
After some more experimentation, it might almost have possibilities for chromakeying... or perhaps not:



David
davidhorman is online now   Reply With Quote
Old 17th January 2010, 13:18   #3  |  Link
tin3tin
Registered User
 
tin3tin's Avatar
 
Join Date: Mar 2005
Posts: 365
Looks very interesting. I guess with animate this could be used for making some nice mask-transitions too. I will give it a try when I get the time.
__________________
DVD slideshow GUI(Freeware).
tin3tin is offline   Reply With Quote
Old 17th January 2010, 14:40   #4  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,327
I've updated the filter to include a new parameter, limit, for the fuse filter. This limits the number of levels of pyramid generated and should result in more pleasing wipes. The effective "width" of the wipe, assuming you start with a hard mask, will be approximately 2^limit, so limit=6 will make your wipe about 64 pixels wide, though that only applies to underlying colour, not the details (which is the whole point of the filter).

David
davidhorman is online now   Reply With Quote
Old 17th January 2010, 15:54   #5  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,327
By tweaking the filter, I've been able to improve the green screen effect to conserve either the background, or the foreground, but unfortunately not both, yet (it may not be possible with this filter):



David
davidhorman is online now   Reply With Quote
Old 21st March 2010, 15:02   #6  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,327
New version

I've updated fusion.dll to fix a couple of bugs, but mainly to include a new filter, fusemask.

Usage:

Code:
fusemask(a,b,exp=0.5)
This creates a binary mask to pass to fuse and can be used to merge two differently exposed images of the same scene, for example:



The parameter exp is a target exposure for the mask, and takes values between 0 and 1 (with 0.5 as the default).

Some of the "harshness" that is evident in fusion's image can be mitigated slightly by blurring the fusemask.

David

Last edited by davidhorman; 21st March 2010 at 15:10.
davidhorman is online now   Reply With Quote
Old 23rd March 2010, 02:00   #7  |  Link
markanini
Registered User
 
Join Date: Apr 2006
Posts: 293
Great work!
markanini is offline   Reply With Quote
Old 27th December 2011, 10:18   #8  |  Link
travolter
Registered User
 
Join Date: Apr 2009
Posts: 136
(newbie here)
Plz, could you write an script example of how to use fuse to boost the levels of underexposed video like in the images of your first post?

Im interested in create HDR vids. Currently Im using the contrast mask technique.. and it provides better results than HDRAGC

Last edited by travolter; 27th December 2011 at 10:21.
travolter is offline   Reply With Quote
Old 28th December 2011, 06:17   #9  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
Hi, can you please more exactly or mathematically express, what exactly fuse() does, because it is not clear to me.

Because afaik the point of creating HDR images is having multiple images of one target at different exposures. Which is in case of video hardly achievable.
redfordxx is offline   Reply With Quote
Old 28th December 2011, 22:38   #10  |  Link
Yellow_
Registered User
 
Join Date: Sep 2009
Posts: 378
Quote:
Originally Posted by redfordxx View Post
Because afaik the point of creating HDR images is having multiple images of one target at different exposures. Which is in case of video hardly achievable.
From post #1

Quote:
I'm not sure it's all that useful for video, but I've written the library now and AviSynth seemed like an interesting way to play with it
Magic Lantern Firmware. Not HDR or multiple exposures of same frame but I'd be interested if you have any suggestions on approach.
http://forum.doom9.org/showthread.ph...89#post1547089
Yellow_ is offline   Reply With Quote
Old 29th December 2011, 16:56   #11  |  Link
tin3tin
Registered User
 
tin3tin's Avatar
 
Join Date: Mar 2005
Posts: 365
A testing script for hdr video shot with the new Magic Lantern EOS hack, with both overlay/blend and Fuse:

Code:
loadplugin("C:\Users\tin2tin\Desktop\Fotos 60D\fusion.dll")
loadplugin("C:\Program Files\DVD slideshow GUI\FFmpegSource\ffms2.dll")
import("C:\Program Files\DVD slideshow GUI\FFmpegSource\ffms2.avs")

v=ffmpegSource2("C:\Users\tin2tin\Desktop\Fotos 60D\MVI_3769.MOV").converttorgb32().ReduceBy2().ReduceBy2()
a=selecteven(v)
b=selectodd(v)

mymask=greyscale(overlay(a,b)).levels(50,1,256,0,256,true)
c=overlay(b,a,mask=mymask,opacity=0.5,greymask=true,mode="blend",pc_range=true)

Stackvertical(a.subtitle("Dark"),c.subtitle("Blend"),fuse(a,b,mymask.invert()).subtitle("Fuse"),b.subtitle("Light"),mymask.subtitle("Mask"))#, fusemask(a,b,exp=0.3)
The footage used: minus.com/mo8xQ556o#1 by DavidJFulde

__________________
DVD slideshow GUI(Freeware).

Last edited by tin3tin; 29th December 2011 at 22:34.
tin3tin is offline   Reply With Quote
Old 29th December 2011, 17:12   #12  |  Link
Yellow_
Registered User
 
Join Date: Sep 2009
Posts: 378
How are you dealing with motion?.

Last edited by Yellow_; 29th December 2011 at 18:48.
Yellow_ is offline   Reply With Quote
Old 29th December 2011, 20:32   #13  |  Link
PALpilot
Registered User
 
Join Date: Dec 2007
Posts: 17
Results are pretty nice. I didn't know this fusion filter even exited. Great!

Combined with the upcoming interframe 2.0 we have everything in one script.

Bart

======= from interframe site ==============

So now interframe interpolates the missing frames from separate even and odd streams.
What would be great is that interframe would use the motionfactors from the opposite frame and tries to recreate that frame as good as possible. I think it could improve quality a lot! Especially on locating walking leg positions or stuff like that.

Please tell me your thoughts on this.
Bart van den Boogaard
Redkitemedia

-------------
SubJunk
December 23rd, 2011 - 11:19

@Bart, it looks great.
Your suggestion is good and it has been developed, along with other upcoming improvements. It is in the private beta stage right now and will be supported in InterFrame 2.0.
I guess it will be ready in a month – or two considering the busy time of year.
Feel free to link back to this blog anywhere you want, traffic is nice

Last edited by PALpilot; 29th December 2011 at 20:34.
PALpilot is offline   Reply With Quote
Old 29th December 2011, 21:18   #14  |  Link
PALpilot
Registered User
 
Join Date: Dec 2007
Posts: 17
UPDATE

Just tried it with both blend and fusion options.
This script is FAST and with little movement very usable WOW!

I went from half an hour workflow when I started to 1 minute now!
PALpilot is offline   Reply With Quote
Old 30th December 2011, 13:08   #15  |  Link
travolter
Registered User
 
Join Date: Apr 2009
Posts: 136
plz anyone testing HDR video can post his script? palpilot and others

Last edited by travolter; 30th December 2011 at 13:19.
travolter is offline   Reply With Quote
Old 30th December 2011, 13:33   #16  |  Link
Yellow_
Registered User
 
Join Date: Sep 2009
Posts: 378
http://blendervse.wordpress.com/2011...lossless-h264/
Yellow_ is offline   Reply With Quote
Old 30th December 2011, 14:24   #17  |  Link
PALpilot
Registered User
 
Join Date: Dec 2007
Posts: 17
I use just this at 720p

v=FFVideoSource("MVI_7980.MOV").converttorgb32().spline36resize(1280,720)
a=selecteven(v)
b=selectodd(v)
mymask=greyscale(overlay(a,b)).levels(50,1,256,0,256,true)
c=overlay(b,a,mask=mymask,opacity=0.5,greymask=true,mode="blend",pc_range=true)

return c
#fuse(a,b,mymask.invert())

=====================
return c = for blend
fuse(a,b,mymask.invert()) is for fusing
switch them by add or remove the # (comment symbol)
Your video has to start with the light frame. Sometimes I need a trim to achieve this.
like v=FFVideoSource("MVI_7980.MOV").trim(0,1).converttorgb32().spline36resize(1280,720)
PALpilot is offline   Reply With Quote
Old 30th December 2011, 15:10   #18  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,376
Quote:
Originally Posted by PALpilot View Post
mymask=greyscale(overlay(a,b)).levels(50,1,256,0,256,true)
Is this right?
Since you are using the default mask and opacity here, doesn't overlay(a,b) just give you b?
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 30th December 2011, 15:15   #19  |  Link
travolter
Registered User
 
Join Date: Apr 2009
Posts: 136
thanks a lot for these scripts examples.. Im anxious to check. I always used contrastmask to create HDR like videos..

http://forum.doom9.org/showthread.php?t=161986
travolter is offline   Reply With Quote
Old 30th December 2011, 16:16   #20  |  Link
PALpilot
Registered User
 
Join Date: Dec 2007
Posts: 17
Quote:
Originally Posted by Gavino View Post
Is this right?
Since you are using the default mask and opacity here, doesn't overlay(a,b) just give you b?
I'm just using the script from tin3tin. I don't understand it completely. I'm just trying it on some footage I shot.

If anyone want some additional HDR footage from Magic Lantern 600d. I can share.
PALpilot is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:37.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.