Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th April 2018, 18:25   #3061  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,406
Quote:
Originally Posted by Selur View Post
@foxyshadis: I get what max_cache_size is meant to do, what I don't get is why setting it to two times your RAM size is the right thing.
It's really not. It can easily eat all the RAM you have when the script is initialized and it takes quite a lot of time for VS to release the excess memory so your computer is pretty much frozen. I've settled down for 10240 on my 16 GB system as I can just manage that with BD to 720p encodes.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 26th April 2018, 04:42   #3062  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 486
Quote:
Originally Posted by DJATOM View Post
I found a bug. Well, not sure if that's really a bug, but might be unwanted behaviour of resizer.
You probably need to provide an script to demonstrate what you really do, if you want the developers to resolve your doubts.
HolyWu is offline   Reply With Quote
Old 26th April 2018, 10:00   #3063  |  Link
DJATOM
Registered User
 
Join Date: Sep 2010
Location: Ukraine, Bohuslav
Posts: 120
Quote:
Originally Posted by HolyWu View Post
You probably need to provide an script to demonstrate what you really do, if you want the developers to resolve your doubts.
Okay. Script below is just stripped version to reproduce my problem.
PHP Code:
import havsfunc as haf
import mvsfunc 
as mvf
import descale 
as dsc

clip 
= <some 1920x1080 clip>

descale_str 0.32
kernel 
'bicubic'
descale_h 540 # 1080/2 to reproduce the problem

clip.width
clip.height
descale_w 
haf.m4((descale_h) / h)
descale_natural dsc.Descale(clipdescale_wdescale_hkernel=kernel)
descale_aa core.resize.Spline36(clipdescale_wdescale_h)
descaled core.std.Merge(clipb=descale_naturalclipa=descale_aaweight=descale_str)

dx w
dy 
h
ux 
descale_w 2
uy 
descale_h 2
rescale 
core.eedi3m.EEDI3(descaledfield=1dh=Truealpha=0.2beta=0.25gamma=1000.0vcheck=3sclip=core.znedi3.nnedi3(descaledfield=1dh=Truensize=0nns=4pscrn=1))
rescale core.std.Transpose(clip=rescale)
rescale core.eedi3m.EEDI3(rescalefield=1dh=Truealpha=0.2beta=0.25gamma=1000.0vcheck=3sclip=core.znedi3.nnedi3(rescalefield=1dh=Truensize=0nns=4pscrn=1))
rescale core.std.Transpose(clip=rescale)
#rescale = core.fmtc.resample(rescale, dx, dy, -0.5, -0.5, ux, uy, kernel='spline36').fmtc.bitdepth(bits=8) # always fixes eedi3 shift, but I need to call bitdepth function, that's not convenient.
rescale core.resize.Spline36(rescaledxdysrc_left=-0.5src_top=-0.5src_width=uxsrc_height=uy# leaves eedi3 shift untouched if rescale clip's width and height matches the source. 
__________________
Me on GitHub | My Telegram
DJATOM is offline   Reply With Quote
Old 26th April 2018, 11:09   #3064  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 486
Quote:
Originally Posted by DJATOM View Post
Okay. Script below is just stripped version to reproduce my problem.
Indeed, the internal resizer doesn't do the shifting if you don't change the dimensions at the same time. I use fmtconv to do the shifting in santiag so never notice this problem.

Here is a much simpler script to reproduce:
PHP Code:
clip core.std.BlankClip(width=640height=360length=1format=vs.YUV420P16color=[655353276832768])
clip core.std.AddBorders(clipleft=40right=40top=60bottom=60)
clip core.std.Interleave([core.fmtc.resample(clipsx=-32sy=-32), core.resize.Spline36(clipsrc_left=-32src_top=-32)]) 
HolyWu is offline   Reply With Quote
Old 26th April 2018, 11:16   #3065  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 421
You could also use edi_rpow2.py https://gist.github.com/YamashitaRen...7524e794779d9c (or just copy the shifting code)
__________________
Search and denoise
ChaosKing is offline   Reply With Quote
Old 26th April 2018, 11:50   #3066  |  Link
DJATOM
Registered User
 
Join Date: Sep 2010
Location: Ukraine, Bohuslav
Posts: 120
Quote:
Originally Posted by ChaosKing View Post
You could also use edi_rpow2.py https://gist.github.com/YamashitaRen...7524e794779d9c (or just copy the shifting code)
zimg way is most likely broken, I didn't checked.
PHP Code:
Y=core.std.ShufflePlanes(clips=clipplanes=0colorfamily=vs.GRAY)
Y=core.resize.Spline36(clip=Y,width=clip.width,height=clip.height,src_left=hshift,src_top=-0.5
clip.width = Y.width, clip.height = Y.height, so resizer is just copying whole plane without doing the job. Another planes should be fixed if colorspace is not YUV444.
__________________
Me on GitHub | My Telegram
DJATOM is offline   Reply With Quote
Old 28th April 2018, 02:49   #3067  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 510
Yay, vapoursynth ffmpeg integration seems to be in its way.

http://ffmpeg.org/pipermail/ffmpeg-d...il/229137.html

I also hold some hope for "for dummies" vapoursynth library build instructions leading to building of a static ffmpeg.exe (including python libs and any other dependencies) ...
hydra3333 is offline   Reply With Quote
Old 28th April 2018, 06:57   #3068  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,627
Quote:
... leading to building of a static ffmpeg.exe (including python libs and any other dependencies) ...
I highly doubt that will happen, since you would probably need to include a whole Python portable environment (or at least a huge portion of it).
To open .vpy-scripts you at least some sort of a Python (runtime) environment.


Cu Selur

Ps.: To open .avs-scripts you need the avisynth.dll (and dependencies).
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 28th April 2018, 07:05   #3069  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 510
Ah. I'd hoped, since youtube-dl manages to do it resulting in a standalone .exe ...

For some, unless maybe it could use vapoursynth portable in the same folder, non-static would be as useful as teats on a bull

Last edited by hydra3333; 28th April 2018 at 07:07.
hydra3333 is offline   Reply With Quote
Old 28th April 2018, 07:26   #3070  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,627
Quote:
Ah. I'd hoped, since youtube-dl manages to do it resulting in a standalone .exe ...
youtube-dl isn't statically compiled it seems to wrap all the dependencies it requires.
Yes once could also wrap a whole hdd this way and create one large binary, but that isn't statically compiling that is basically the same thing Mac does with it's .app files.
(Provide mostly standalone packages which are basically a folder structure which contains dependencies&co.)

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 30th April 2018, 00:36   #3071  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 421
Is it possible to compare the value of f.props.PlaneStatsAverage in a FrameEval function with the previous and the next frame (n-1 and n+1)? Sould I use get_frame_async() for that?

Code:
def g(n, f, clip, d_clip):
	clip=clip.text.Text("current: "+ f.props.PlaneStatsAverage)
	clip=clip.text.Text("\nPrev: "+ ???)
	clip=clip.text.Text("\n\nNext: "+ ???)
		
	return clip
clip = clip.std.FrameEval(functools.partial(g, d_clip=d_clip, clip=clip), prop_src=x)
Edit
This works but feels more like a hack.
Code:
x= clip.std.PlaneStats()
x1= clip.std.PlaneStats().std.DuplicateFrames(0) #prev
x2= clip.std.PlaneStats().std.Trim(1) #next

clip = clip.std.FrameEval(functools.partial(g, d_clip=d_clip, clip=clip), prop_src=[x, x1, x2])
__________________
Search and denoise

Last edited by ChaosKing; 30th April 2018 at 01:11.
ChaosKing is offline   Reply With Quote
Old 1st May 2018, 22:39   #3072  |  Link
amayra
Quality Checker
 
amayra's Avatar
 
Join Date: Aug 2013
Posts: 206
Quote:
Python development mode
A new command-line switch for the Python interpreter, -X, lets the developer set a number of low-level options for the interpreter. With Python 3.7, the option -X dev enables “development mode,” a slew of runtime checks that normally have a big impact on performance, but are useful for a developer during the debugging process
as vapoursynth user is this benefits me ?
__________________
I love Doom9
amayra is offline   Reply With Quote
Old 1st May 2018, 22:44   #3073  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,859
Quote:
Originally Posted by ChaosKing View Post
Is it possible to compare the value of f.props.PlaneStatsAverage in a FrameEval function with the previous and the next frame (n-1 and n+1)? Sould I use get_frame_async() for that?

Code:
def g(n, f, clip, d_clip):
	clip=clip.text.Text("current: "+ f.props.PlaneStatsAverage)
	clip=clip.text.Text("\nPrev: "+ ???)
	clip=clip.text.Text("\n\nNext: "+ ???)
		
	return clip
clip = clip.std.FrameEval(functools.partial(g, d_clip=d_clip, clip=clip), prop_src=x)
Edit
This works but feels more like a hack.
Code:
x= clip.std.PlaneStats()
x1= clip.std.PlaneStats().std.DuplicateFrames(0) #prev
x2= clip.std.PlaneStats().std.Trim(1) #next

clip = clip.std.FrameEval(functools.partial(g, d_clip=d_clip, clip=clip), prop_src=[x, x1, x2])
That's the right way.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 1st May 2018, 23:03   #3074  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,602
@ amayra:

Users will probably not benefit directly from debugging switches, because users will usually prefer the fastest possible execution. You may have read that these additional runtime checks will "have a big impact on performance" (see your quote), which means that it will run a lot slower and report a lot of warnings, even of less probable conditions.

But if a developer can use this mode to discover risky code and fix bugs (possibly even before they cause noticeable issues), then users may benefit indirectly from an advanced release build.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 2nd May 2018, 14:16   #3075  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 421
@Myrsloik
Ok, thx.
I found a strange behaviour while playing with FrameEval func.

The very same function gives different output if it's called within FrameEval.
Is it a bug or did I miss something?
Here's an animated gif of my problem (correct output is in the middle):


And my script (You'll need also Single Precision MVTools https://github.com/IFeelBloated/vapo...h-mvtools-sf):
Code:
import sys
import os

import functools
import vapoursynth as vs

import mvsfunc as mvf #https://github.com/HomeOfVapourSynthEvolution/mvsfunc/blob/master/mvsfunc.py
import havsfunc as haf
import mvmulti as mv #https://github.com/IFeelBloated/vapoursynth-mvtools-sf/blob/master/src/mvmulti.py
import fvsfunc as fvf #https://github.com/Irrational-Encoding-Wizardry/fvsfunc/blob/master/fvsfunc.py

core = vs.get_core()

def myFilter(clip):
	pre = haf.SMDegrain(clip, tr=2, pel=2, contrasharp=False, thSAD = 1200, thSADC = 1200, prefilter=2)
	pre = pre.flux.SmoothT(temporal_threshold=6)
	pre = haf.DitherLumaRebuild(pre, s0=1)
	pre = fvf.Depth(pre, 32, range_in=1)
	
	clip = mvf.Depth(clip, 32)
	
	super = core.mvsf.Super(pre,16,16,1,0)
	vectors = mv.Analyze(super, blksize=16, overlap=8, search=4, tr=6)
	super = core.mvsf.Super(clip,16,16,1,1)
	blur_clip = mv.DegrainN(clip, super, vectors, thsad=600, tr=6)
	
	grain = blur_clip.grain.Add(var=1200.0, constant=True)#.std.BoxBlur(hradius=1, hpasses=1, vradius=1, vpasses=1)
	#return grain
	diff_clip = core.std.Expr([clip, blur_clip], 'x y - abs').std.Inflate(threshold=200/255).std.Inflate(threshold=200/255)
	mask_clip = diff_clip.std.Binarize(threshold=[3.3/219, 3.3/224], v0=0, v1=80/255)
		
	clip = core.std.MaskedMerge(clipa=blur_clip, clipb=grain, mask=mask_clip)
	
	return mvf.Depth(clip, 8)
	
	

clip = core.std.BlankClip(format=vs.YUV420P8, width=120*2, height=80*2, length=100, color=[206,235,135])
g_static  = clip.grain.Add(var=100.0, constant=True)
g_dynamic = clip.grain.Add(var=100.0, constant=False)


clip = core.std.StackHorizontal([
	core.std.StackVertical([g_static, g_static]), 
	core.std.StackVertical([g_static, g_dynamic])
	])
goal = core.std.StackHorizontal([
	core.std.StackVertical([g_static, g_static]), 
	core.std.StackVertical([g_static, g_static])
	])
orig=clip




def asd(n, c):
	return myFilter(c)
	
feval = clip.std.FrameEval(functools.partial(asd, c=clip))

def comp(a, b, crop=0):
	return core.std.StackHorizontal([
		core.std.CropRel(a, crop,crop,0,0), \
		core.std.CropRel(b, crop,crop,0,0), \
		])
		
nofeval = myFilter(clip)

clip = comp(feval.text.Text("Function inside FrameEval").std.AddBorders(right=2), nofeval.text.Text("No FrameEval").std.AddBorders(right=2), crop=0)
clip = comp(clip, orig.text.Text("unfiltered"), crop=0)

clip.set_output()
__________________
Search and denoise

Last edited by ChaosKing; 2nd May 2018 at 14:22.
ChaosKing is offline   Reply With Quote
Old 2nd May 2018, 14:22   #3076  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,859
Quote:
Originally Posted by ChaosKing View Post
@Myrsloik
Ok, thx.
I found a strange behaviour while playing with FrameEval func.

The same function gives different output if it's called within FrameEval.
Is it a bug or did I miss something?
Here's a animated gif of my problem (correct output is in the middle):


And my script (You'll need also Single Precision MVTools https://github.com/IFeelBloated/vapo...h-mvtools-sf):
...
Should be identical in theory, yes. Unless some filter stores temporal state incorrectly. Then you get this. Gradually remove each filter and see when the output is once again identical, then report which filter it was. I'm going to bet mvtools or fluxsmooth right away.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 2nd May 2018, 14:31   #3077  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 421
Seems it only happens if mvtools-sf is in the filter chain.
Edit:
Is there a way (hack) to avoid this problem without modifying mvtools? I mean except putting it outside of FrameEval, bcs it's very slow and I need it only for many short scenes.

I will report this bug to feisty2 but who knows when and if he's gonna fix it.
__________________
Search and denoise

Last edited by ChaosKing; 2nd May 2018 at 14:50.
ChaosKing is offline   Reply With Quote
Old 2nd May 2018, 15:31   #3078  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 486
Generally I don't recommend invoking filters within FrameEval unless unavoidable, because that introduces extra overhead.

PHP Code:
from vapoursynth import core
import vapoursynth 
as vs
import functools

def frame_eval
(nclip):
    return 
core.misc.AverageFrames(clipweights=[1] * 5)

clip core.std.BlankClip(width=720height=480format=vs.YUV420P8length=10000)

#clip = core.misc.AverageFrames(clip, weights=[1] * 5)
# I get 1014.60 fps

#clip = core.std.FrameEval(clip, functools.partial(frame_eval, clip=clip))
# I get 566.09 fps

clip.set_output() 
You should assign the filtered result to a variable outside FrameEval and then pass it in via functools.partial() as additional argument if possible.
HolyWu is offline   Reply With Quote
Old 2nd May 2018, 15:39   #3079  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 421
Quote:
Originally Posted by HolyWu View Post
Generally I don't recommend invoking filters within FrameEval unless unavoidable, because that introduces extra overhead.

PHP Code:
from vapoursynth import core
import vapoursynth 
as vs
import functools

def frame_eval
(nclip):
    return 
core.misc.AverageFrames(clipweights=[1] * 5)

clip core.std.BlankClip(width=720height=480format=vs.YUV420P8length=10000)

#clip = core.misc.AverageFrames(clip, weights=[1] * 5)
# I get 1014.60 fps

#clip = core.std.FrameEval(clip, functools.partial(frame_eval, clip=clip))
# I get 566.09 fps

clip.set_output() 
You should assign the filtered result to a variable outside FrameEval and then pass it in via functools.partial() as additional argument if possible.
Why didn't I think of that... it works perfectly.
__________________
Search and denoise
ChaosKing is offline   Reply With Quote
Old 2nd May 2018, 16:24   #3080  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 486
Sorry, the above sample was invalid. Because one called FrameEval but the other didn't, the comparison was not under fair condition. So here is another proper sample.

PHP Code:
from vapoursynth import core
import vapoursynth 
as vs
import functools

def frame_eval1
(nfclip):
    if 
f.props['_Combed']:
        return 
core.std.BoxBlur(clip)
    else:
        return 
clip

def frame_eval2
(nfblurredorig):
    if 
f.props['_Combed']:
        return 
blurred
    
else:
        return 
orig

blank 
core.std.BlankClip(width=720height=480format=vs.YUV420P8length=5000)
non_combed core.std.SetFrameProp(blankprop='_Combed'intval=0)
combed core.std.SetFrameProp(blankprop='_Combed'intval=1)
clip core.std.Interleave([non_combedcombed])

#clip = core.std.FrameEval(clip, eval=functools.partial(frame_eval1, clip=clip), prop_src=clip)
# I get 1033.63 fps

#clip = core.std.FrameEval(clip, eval=functools.partial(frame_eval2, blurred=core.std.BoxBlur(clip), orig=clip), prop_src=clip)
# I get 3073.43 fps

clip.set_output() 
HolyWu is offline   Reply With Quote
Reply

Tags
speed, vaporware, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:19.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.