Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th January 2011, 17:42   #1  |  Link
zee944
Registered User
 
Join Date: Apr 2007
Posts: 240
an idea for fixing haloed, chroma bleeding edges

I have a source with ugly, chroma bleeding, haloed, and blurry edges I'm unable to fix for quite a long time now. Here's a sample picture from it (fragments of frames actually, no resizing).

It may not look so bad from the pictures, but guaranteed to look really bad after adding any sharpening or heavier color correction.

MergeChroma(aWarpSharp(depth=32,blurlevel=1,thresh=0.5,cm=1)) does a help a little; YAHR() also helps a bit, but all in all, the root problem is still there, the edge problems are more apparent then ever after further tweaking:



I've been playing around with chroma shifters and dehaloers, with little to no results. The only thing that really fixed the edges was aWarpSharp(depth=32,blurlevel=2,thresh=0.75). I just loved those edges but the result was obviously unusable due to the heavy deformations and the lack of details.

I'm desperate by now. I thought of something to have these great edges with the original geometry and details:

1. take a frame, use aWarpSharp to create a deformed frame with perfect edges

2. motion compensate the deformed frame to the original frame

3. copy the perfect edges onto the original frame


Here's my script so far (sample m2v):

Code:
LoadPlugin("mvtools2[2.5.11.1].dll")
LoadPlugin("aWarpSharp[b1].dll")			# chroma-probléma javításához kell
LoadPlugin("mt_masktools[2.0a30].dll")

DirectShowSource("TD_sample.m2v")
ConvertToYV12()

AddBorders(16,0,16,0)
MergeChroma(aWarpSharp(depth=32,blurlevel=1,thresh=0.5,cm=1))		# pre-fixing a chroma noise
Crop(16,0,-16,0)

original=last

# Instead of chroma bleeding, haloed and blurry edges we're creating good edges, but cartoonish and deformed look:
awarpsharped=original.aWarpSharp(depth=32,blurlevel=2,thresh=0.99)

# Getting back to the original shapes with motion compensation to have good and undeformed edges, but still cartoonish look:
Interleave(original,awarpsharped)
super=MSuper(last)
forward_vectors = MAnalyse(super, isb=true)
MFlow(super,forward_vectors)
SelectEvery(2,0)
fixedawarpsharped=last		

edgemask=original.MergeLuma(mt_edge("sobel",7,7,5,5))	
# Extending and blurring the mask to have some transition at overlaying:
edgemask=edgemask.mt_inflate().Blur(1.58).Blur(1.58)

# Bad edges replaced with good edges on the original image:
endresult=Overlay(original,fixedawarpsharped,mask=edgemask)

StackVertical(original.Crop(170,70,-170,-70).subtitle("original"),endresult.Crop(170,70,-170,-70).subtitle("end result"))
Script result:




The script shows only the general idea, the result doesn't look good due to the imperfect motion compensating and poorly selected edges. It's cartoonish and the sharpness difference between the tweaked and non-tweaked edges are not handled either (that'll be a much later step I guess).

Questions:

1. Why the motion compensated frames ("fixedawarpsharped") seem not fully compensated? Like if the motion on it was halfway between the "original" and the "awarpsharped" motion. Am I miss something here?

2. How could I create a better edgemask to select only the "important" (problematic) edges?

3. The compensated edges look somewhat pixelated and unstable, can MVTools2 produce better result with different settings?

4. How to "cartoonize" the original clip to look very similar to the awarpsharped one (for better motion estimation)?

Could you help me getting the best out of this method? It really looks promising to me, I hope it'll fix my video eventually, and could be used for other sources too. Any tweaks, ideas and answers are appreciated
zee944 is offline   Reply With Quote
Old 24th January 2011, 19:37   #2  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Regarding the motion-compensation part: Reduce blocksize to 4, reduce hierarchical levels to 2 or even 1, and make a small-range exhaustive search. (The only thing you want to compensate is the warpsharp deformations. Hence, the wanted compensation target should be very close to the starting pixel, usually not more than 1~2 pixels away.)

It might pay off to apply some blurring to the motionsearch clip - after all, the warpsharpen'ed edges are sharper than the original edges. Would be better if, in the searchclip, both clips have a similar level of sharpness (or blurryness) .... else, Manalyse would need to match a "sharp" edge onto a "blurry" edge, which obviously isn't ideal.


edit: Regarding detail loss: make a comparison where the original input has "more edge" than the result. Those are (probably) the areas where merging-back the original is appropriate.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 24th January 2011 at 19:49.
Didée is offline   Reply With Quote
Old 26th January 2011, 18:53   #3  |  Link
zee944
Registered User
 
Join Date: Apr 2007
Posts: 240
All of your suggestions helped, thanks for them. I've ended up with this script:

Code:
Function mcaWarpSharp(clip source, int "supsamp")
{# by z. ndmn.	dependencies: MVTools2, masktools, aWarpSharp[20090619]
supsamp = default(supsamp, 4)	# supsamp=2 recommended for testing, supsamp=4 for encoding.
source
# for no supersampling (supsamp=1): (thresh=255,blur=3,type=1,depth=8,chroma=4)
# for 4x supersampling (supsamp=4): (thresh=192,blur=36,type=1,depth=96,chroma=4)
WarpedEdges=last.Lanczos4Resize(width*supsamp,height*supsamp).\
	aWarpSharp2(thresh=int(255*pow(supsamp,-0.207)),blur=3*int(pow(supsamp,1.7925)),type=1,depth=8*int(pow(supsamp,1.7925)),chroma=4)

# Getting back to the original shapes with motion compensation to have good and undeformed edges, but still cartoonish look:
Interleave(last.Lanczos4Resize(width*supsamp,height*supsamp),WarpedEdges)
blurredclip=GaussResize(width*2,height*2,p=32*pow(supsamp,-2)).GaussResize(width,height,p=32*pow(supsamp,-2))
super=MSuper(blurredclip, levels=1)
super2=MSuper(last, levels=1)
forward_vectors = MAnalyse(super, blksize=4, levels=1, search=3, searchparam=5, isb=true)
MFlow(super2,forward_vectors)
SelectEvery(2,0)
Lanczos4Resize(width/supsamp,height/supsamp)

# Replacing 'bad' edges with 'good' edges on the original image:
edgemask=source.mt_edge("min/max").GreyScale()
Overlay(source,last,mask=edgemask)
#Overlay(last,                      last.Blur(0.5),mask=edgemask)	# a little blurring on the sharp new edges...
#Overlay(last.sharpen(0.5),last,               mask=edgemask)	# ...or sharpening everywhere else to make the image consistent.
Return(last)
}
The motion estimation is nearly perfect now, the only thing in the current workflow that could be better is the edge masking. But all in all, the end result seems quite good.
zee944 is offline   Reply With Quote
Old 28th November 2011, 21:28   #4  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,567
Can somebody help me to speed up this script? I tweaked mcawarpsharp a bit for speed and control but this run at around 0.15fps on a 720x480 clip with 96000 frames. I think the bottleneck is at mflow in the 4x supsampled clip but I'd like to know if I can speed up that part or others as well without using a smaller supsampling.

Code:
setmtmode(5,2)
ffvideoSource("FFV1intermediary.avi")
a=last
setmtmode(2,2)


white=70
mt_edge("min/max",thY1=0,thY2=255).removegrain(11).mt_expand.mt_expand.mt_lut("x "+string(white)+" > 255 x "+string(white)+" / 255 * ?")

BicubicResize (180,120, 1, 0)
TemporalSoften(10,30,0,15,2)
BicubicResize (720,480, 1, 0)
global edgemask=last

deconv_v =  "-3 10 4 -4 8 -6 100 -6 8 -4 4 10 -3"
mt_merge(a,a.mt_convolution(horizontal="1",vertical=deconv_v, Y=3, U=2, V=2),edgemask,u=4,v=4)

MCaWarpSharp4(4,2,0.6,true,640,360)

smdegrain(tr=3,thSAD=400,hpad=0,vpad=0,chroma=false,plane=0)



function MCaWarpSharp4(clip source, int "supsamp",int "post", float "PPstr", bool "fast", int "w", int "h"){

supsamp = default(supsamp, 4)	# supsamp=2 recommended for testing, supsamp=4 for encoding.
post    = default(post, 4)      # mix the warpsharped edges with original footage thru blur and sharpen post processing
str     = default(PPstr,1.0)
fast    = default(fast,true)
w       = default(w,source.width())
h       = default(h,source.height())


var=32*pow(supsamp,-2)
w1=source.width()*supsamp
h1=source.height()*supsamp

source
sourceSup=fast ? bicubicResize(w1,h1,b=0.0,c=0.5) : nnedi3_rpow2(rfactor=supsamp,cshift="spline64resize",nns=4,qual=2)

WarpedEdges=sourceSup.aWarpSharp2(
\ thresh=int(255*pow(supsamp,-0.207)),
\ blur=3*int(pow(supsamp,1.7925)),
\ type=1,depth=8*int(pow(supsamp,1.7925)),chroma=4)

Interleave(sourceSup,WarpedEdges)

blurredclip=GaussResize(w1*2,h1*2,p=var).GaussResize(w1,h1,p=var)

super  =          MSuper(blurredclip, levels=1, pel=fast?1:2, sharp=fast?0:2)
super2 =          MSuper(levels=1, pel=fast?1:2, sharp=fast?0:2)
forward_vectors = MAnalyse(super, blksize=4, levels=1, search=3, searchparam=5, isb=true)
                  MFlow(super2,forward_vectors)

SelectEvery(2,0)
fast ? bicubicResize(w,h,b=0.0,c=0.5) : Spline36Resize(w,h)

# Replacing 'bad' edges with 'good' edges on the original image
source    = post==1 && (Defined(w) || Defined(h)) ? (fast ? source.bicubicResize(w1,h1,b=0.0,c=0.5) : source.Spline36Resize(w1,h1)) : source
#~ edgemask  = source.mt_edge("min/max").blur(1.0)

Assert  ((post >= 1 && post <= 4 )  ? true : false, chr(10) + "'post' have not a correct value! [1,2,3,4]" + chr(10))
post =   (post==1) ? mt_merge(source,last,edgemask.bilinearresize(w,h),luma=true)
    \  : (post==2) ? mt_merge(Blur(min(str*0.5,1.58)),edgemask.bilinearresize(w,h),luma=true)
    \  : (post==3) ? mt_merge(LSFmod(defaults=fast?"fast":"slow",strength=int(str*20),edgemode=0,soothe=true,ss_x=1.0,ss_y=1.0),last,edgemask.bilinearresize(w,h),luma=true)
    \  : (post==4) ? mt_merge(LSFmod(defaults=fast?"fast":"slow",strength=int(str*20),edgemode=0,soothe=true,ss_x=1.0,ss_y=1.0),Blur(min(str*0.5,1.58)),edgemask.bilinearresize(w,h),luma=true)
    \  :  blankclip(source, width=width, height=120).SubTitle("post = " + string(post),text_color=$FFFFFF,font="COURIER NEW",size=12,x=320,y=90)
return post}


edit: I tweaked the manalyse portion of the mcawarpsharp4 function with the new mscalevectors from cretindesalpes, but results are no good, I can't go sub 4 for blksize so I don't know if that's the reason, also mode=2 doesn't work, so I couldn't check.

Code:
function MCaWarpSharp4(clip source, int "supsamp",int "post", float "PPstr", bool "fast", int "w", int "h"){

supsamp = default(supsamp, 4)	# supsamp=2 recommended for testing, supsamp=4 for encoding.
post    = default(post, 4)      # mix the warpsharped edges with original footage thru blur and sharpen post processing
str     = default(PPstr,1.0)
fast    = default(fast,true)
w       = default(w,source.width())
h       = default(h,source.height())


var=32*pow(supsamp,-2)
w1=source.width()*supsamp
h1=source.height()*supsamp

source
sourceSup=fast ? bicubicResize(w1,h1,b=0.0,c=0.5) : nnedi3_rpow2(rfactor=supsamp,cshift="spline64resize",nns=4,qual=2)

WarpedEdges=sourceSup.aWarpSharp2(
\ thresh=int(255*pow(supsamp,-0.207)),
\ blur=3*int(pow(supsamp,1.7925)),
\ type=1,depth=8*int(pow(supsamp,1.7925)),chroma=4)

Interleave(sourceSup,WarpedEdges)

blurredclip=GaussResize(w1*2,h1*2,p=var).GaussResize(720,480,p=32*pow(1,-2))

super  =          MSuper(blurredclip, levels=1, pel=fast?1:2, sharp=fast?0:2,hpad=0,vpad=0)
super2 =          MSuper(levels=1, pel=fast?1:2, sharp=fast?0:2,hpad=0,vpad=0)
forward_vectors = MAnalyse(super, blksize=4, levels=1, search=3, searchparam=5, isb=true).MScaleVect(4)
                  MFlow(super2,forward_vectors)

SelectEvery(2,0)
fast ? bicubicResize(w,h,b=0.0,c=0.5) : Spline36Resize(w,h)

# Replacing 'bad' edges with 'good' edges on the original image
source    = post==1 && (Defined(w) || Defined(h)) ? (fast ? source.bicubicResize(w1,h1,b=0.0,c=0.5) : source.Spline36Resize(w1,h1)) : source
#~ edgemask  = source.mt_edge("min/max").blur(1.0)

Assert  ((post >= 1 && post <= 4 )  ? true : false, chr(10) + "'post' have not a correct value! [1,2,3,4]" + chr(10))
post =   (post==1) ? mt_merge(source,last,edgemask.bilinearresize(w,h),luma=true)
    \  : (post==2) ? mt_merge(Blur(min(str*0.5,1.58)),edgemask.bilinearresize(w,h),luma=true)
    \  : (post==3) ? mt_merge(LSFmod(defaults=fast?"fast":"slow",strength=int(str*20),edgemode=0,soothe=true,ss_x=1.0,ss_y=1.0),last,edgemask.bilinearresize(w,h),luma=true)
    \  : (post==4) ? mt_merge(LSFmod(defaults=fast?"fast":"slow",strength=int(str*20),edgemode=0,soothe=true,ss_x=1.0,ss_y=1.0),Blur(min(str*0.5,1.58)),edgemask.bilinearresize(w,h),luma=true)
    \  :  blankclip(source, width=width, height=120).SubTitle("post = " + string(post),text_color=$FFFFFF,font="COURIER NEW",size=12,x=320,y=90)
return post}

Last edited by Dogway; 29th November 2011 at 04:17.
Dogway is offline   Reply With Quote
Old 29th November 2011, 13:06   #5  |  Link
*.mp4 guy
Registered User
 
*.mp4 guy's Avatar
 
Join Date: Feb 2004
Posts: 1,350
... While not strictly an optimization of the posted script, this probably fits the bill. I've been using it for a while, never thought to post it.

Soft thresholded warpsharp:
Code:
Function Twarp(Clip c, int "thresh", float "depth", int "blurlevel", float "wthresh")
{

	Thresh = Default(Thresh, 8)
	depth = Default(depth, 5)
	blurlevel = Default(blurlevel, 1)
	wthresh = Default(wthresh, 0.99)
		
		B1 = C.awarpsharp(depth=depth, blurlevel=blurlevel, thresh=wthresh)
			Mt_LutXY(C, b1, " X Y - abs X Y - abs * X Y -  * X Y - abs X Y - abs * "+string(thresh)+" + / 128 +", u=1, v=1)
	
Mt_AddDiff(last, b1, u=1, v=1)
}
Thresh is the strength of the limiting, all other parameter map to awarpsharp equivalents.

Defaults are very conservative, intended to reduce haloing without any other effects. For stronger settings, try something such as: Twarp(64, 12, 2, 0.7)

Re-writing for new warpsharp should be very simple if it bothers anyone, but I can't be bothered atm.
*.mp4 guy is offline   Reply With Quote
Old 30th November 2011, 01:29   #6  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 1,567
Probably something around this?

Code:
Function Twarp(Clip c, int "thresh", int "depth", int "blurlevel", int "wthresh")
{

	Thresh = Default(Thresh, 8)
	depth = Default(depth, 3)
	blurlevel = Default(blurlevel, 8)
	wthresh = Default(wthresh, 253)
		
		B1 = C.awarpsharp2(depth=depth, blur=blurlevel, type=1,chroma=3,thresh=wthresh)
			Mt_LutXY(C, b1, " X Y - abs X Y - abs * X Y -  * X Y - abs X Y - abs * "+string(thresh)+" + / 128 +", u=1, v=1)
	
Mt_AddDiff(last, b1, u=1, v=1)}
Twarp(64, 6, 16, 179)


It didn't perform too nice, it deformed too much (I'm using live footage), or that is my impression, but thanks for sharing I think I will make use of it eventually when I test more on other sources.


Dogway is offline   Reply With Quote
Reply

Tags
awarpsharp, chroma, dehaloer, edges

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:55.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.