Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th January 2020, 00:05   #1  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Apply filters with local gradient

There's a thread on the Avisynth forum about how to apply a filter using different arguments controlled by a gradient mask. The author asked to see my VapourSynth implementation of that idea so I'm posting it here.

Code:
import vapoursynth as vs
import adjust
from functools import partial
core = vs.core
	
# creates a 8-bit gray gradient mask
# set steps to a value 2-254 to have less than 255 different gray values
def makeMask(clip, steps=None):
	format = vs.GRAY8
	maxval = 255
		
	stripes = []
	height = steps or clip.height
	for i in range(height):
		alpha = maxval * i / (height-1)
		stripes.append(core.std.BlankClip(format=format, color=alpha, width=clip.width, height=1))

	gradientMask = core.std.StackVertical(stripes)	
	
	if gradientMask.height < clip.height:
		gradientMask = core.resize.Point(gradientMask, width=clip.width, height=clip.height)	
	
	return gradientMask


# apply gradient mask + function, assumes 8-bit mask, returns result clip as YUV444P8
def doGradient(mask, clip, gradientFunc):

	# convert video to YUV444P8 in order to access per pixel chroma
	clip = core.resize.Bilinear(clip, clip.width, clip.height, format=vs.YUV444P8)

	# this function runs for every frame
	#   n = frame number
	#   f = frame properties
	#   mask = mask clip
	#   src = source clip to apply the effect to
	#   i = current mask value (0-255)
	def applyFunc(n, f, mask, src, i):
		# skip if the mask has all zeroes (max value is zero)
		if f.props['PlaneStatsMax'] > 0:
			# calculate the effect and return it for the current frame, function gets called with argument ratio between 0.0 - 1.0
			return gradientFunc(src, i/255)
		
		# no effect applied, return source clip as is
		return src

	# create final "canvas" where final result will be composited to		
	final = core.std.BlankClip(clip, clip.width, clip.height)
	
	# iterate through all possible mask values (0-255)
	for i in range(256):
		# create mask with 255 where original mask has value i, zero otherwise
		m = core.std.Expr(mask, f'x {i} = 255 0 ?')
		
		# calculate basic statistics for the mask (average, min and max brightness), will be used to skip unnecessary processing
		m = core.std.PlaneStats(m, plane=0)

		# evaluate applyFunc() for every frame, returns effect applied to whole frame if mask m is not all zeroes
		gradient_clip = core.std.FrameEval(final, partial(applyFunc, mask=m, src=clip, i=i), prop_src=m)	
		
		# create a 3-channel mask to extract the parts where mask m has value 255
		yuv_mask = core.std.ShufflePlanes(clips=[m,m,m], planes=[0,0,0], colorfamily=vs.YUV)	
		
		# draw the contribution of mask value i to canvas (take pixels from gradient_clip where mask = 255)
		final = core.std.Expr([final, gradient_clip, yuv_mask], 'z 255 = y x ?')
	
	# return final composition
	return final


# example function: blur
def blur_func(clip, ratio):
	clip = core.std.BoxBlur(clip, hradius=ratio*40, hpasses=1)
	#clip = core.std.BoxBlur(clip, vradius=ratio*40, vpasses=1)
	return clip


# example function: tweak (brightness, contrast)
def tweak_func(clip, ratio):
	clip = adjust.Tweak(clip, bright=-ratio*100.0, cont=1.0 + ratio)
	#clip = adjust.Tweak(clip, cont=1.0 + ratio*0.5)
	return clip



# load source
src = core.ffms2.Source(source = 'source.avi')

# create a static gradient mask (you could use a video for a mask, too)
mask = makeMask(src, steps=None)

# apply mask + given function to source
result = doGradient(mask, src, blur_func)

# show results
result.set_output()
Hopefully there are some ideas that could be used in the Avisynth version (which currently has some memory issues).

This version runs about 1 fps on a 1080x576 source video using a mask with 256 different values and 5-6 fps using a mask with 16 different values. The mask could be created from a video clip as well, this example just creates a static gradient.
zorr is offline   Reply With Quote
Old 17th January 2020, 01:07   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
WOW!, thanks Zorr, maybe I gotta try VS one day.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 17th January 2020, 10:10   #3  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 201
Hi zorr.

Thank you very much from my side too.

This looks clean and neat, just the way I thought the alg could be.

But I would like to do a little correction:
I'm new to Avisynth either, but it seems the mem issue you mentioned
is not caused by an error of the script, but by an error of Avisynth itself.
nji is offline   Reply With Quote
Old 17th January 2020, 12:16   #4  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
So basically you have 256 (or less) input clips and then you want to effectively grab pixels from different ones depending on the value of a mask?

You really should look into writing a proper filter because then it'll run a lot faster due to greatly improved parallelism.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 17th January 2020, 18:43   #5  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
i tried the script, 760 MB RAM usage on initial load, and it goes over 4 GB in just seven frames in
lansing is offline   Reply With Quote
Old 17th January 2020, 22:34   #6  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 201
... into disappearing infinity?
nji is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.