Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th December 2012, 01:40   #21  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 643
Quote:
Originally Posted by kolak View Post
On my laptop anything different than ultra fast does not work at all. Which plugins are the best to use with vs- I use Vit's modified version (but not the one for avisynth 2.6).
What exactly "does not work" means? Do you mean the output frame is the same as if QTGMC had no effect? Or is there any error message? If so, you should post it as well. The plugins I use mostly are the official ones, except nnedi3 and eedi3 are native VapourSynth plugin. I didn't use Vit's modified version, so no idea if it works with VapourSynth.

EDIT: Did you try replacing this new vsvfw.dll? Though it maybe have nothing to do with your problem.

If in doubt, just try encoding a small range of your video and see whether the encoding could finish, and whether the frames of the encoded file has problems or not.

Last edited by HolyWu; 4th December 2012 at 07:46.
HolyWu is offline   Reply With Quote
Old 4th December 2012, 01:57   #22  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,410
No- Vdub shows first frame and than does nothing- CPU stays at 0. I can't jump into any frame etc- just hangs.
I will try unmodified version, see if it changes anything. Looks like ultrafast uses different plugins than all others.
kolak is offline   Reply With Quote
Old 8th December 2012, 23:54   #23  |  Link
Revgen
Registered User
 
Join Date: Sep 2004
Location: Near LA, California, USA
Posts: 1,545
I found out how to increase WinXP's memory limit to 3GB. That helps to make the process to go longer, but it still crashes after using over 2.2 GB of memory. How much memory does QTGMC need anyway?

Also I tried to use Vapoursynth's set_max_cache_size() but couldn't figure out how to use it. So then I tried using vs.Core(threads=2) to see if it would use less memory instead of utilizing all 4 CPU cores. The program hangs and doesn't do anything. Ditto with threads=1. The lowest I can go is threads=3, and even that reaches to 2.2gb and crashes. I tried the vsvfw.dll too. No luck.

I tried Vdub, x264, and FFPlay. Same results on all.

I'm wondering if I have a bad version of a plugin. Is there a way to monitor which plugin is utlizing the most memory?
__________________
Pirate: Now how would you like to die? Would you like to have your head chopped off or be burned at the stake?

Curly: Burned at the stake!

Moe: Why?

Curly: A hot steak is always better than a cold chop.
Revgen is offline   Reply With Quote
Old 9th December 2012, 01:29   #24  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,017
Quote:
Originally Posted by Revgen View Post
I found out how to increase WinXP's memory limit to 3GB. That helps to make the process to go longer, but it still crashes after using over 2.2 GB of memory. How much memory does QTGMC need anyway?

Also I tried to use Vapoursynth's set_max_cache_size() but couldn't figure out how to use it. So then I tried using vs.Core(threads=2) to see if it would use less memory instead of utilizing all 4 CPU cores. The program hangs and doesn't do anything. Ditto with threads=1. The lowest I can go is threads=3, and even that reaches to 2.2gb and crashes. I tried the vsvfw.dll too. No luck.

I tried Vdub, x264, and FFPlay. Same results on all.

I'm wondering if I have a bad version of a plugin. Is there a way to monitor which plugin is utlizing the most memory?
There's a bug in vs that makes the memory limit 1024 times bigger than it should be meaning that the default max is 1TB. As a temporary workaround try setting the max cache size to 1. Will be fixed in the next release which should be out sometime next week.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 9th December 2012, 02:15   #25  |  Link
Revgen
Registered User
 
Join Date: Sep 2004
Location: Near LA, California, USA
Posts: 1,545
I haven't been able to figure out how to do set_max_cache_size().

If I do it this way

Code:
import vapoursynth as vs
import sys
import havsfunc

core = vs.Core().set_max_cache_size(1)
haf = havsfunc.HAvsFunc(core)
#plugins#
I get a "avisynth open failure; Python exception. 'int' object has no attribute 'avs'" error.

I'm not sure exactly where it's supposed to go.

I guess I'll wait until next week for the new release.
__________________
Pirate: Now how would you like to die? Would you like to have your head chopped off or be burned at the stake?

Curly: Burned at the stake!

Moe: Why?

Curly: A hot steak is always better than a cold chop.

Last edited by Revgen; 9th December 2012 at 02:35.
Revgen is offline   Reply With Quote
Old 9th December 2012, 02:34   #26  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 280
Code:
core = vs.Core()
core.set_max_cache_size(1)
Are_ is offline   Reply With Quote
Old 9th December 2012, 04:10   #27  |  Link
Revgen
Registered User
 
Join Date: Sep 2004
Location: Near LA, California, USA
Posts: 1,545
Thanks Are. No error messages this time, but it's still exhibiting the same behavior. Looks like I'll just have to wait til the next release to see if the situation improves.
__________________
Pirate: Now how would you like to die? Would you like to have your head chopped off or be burned at the stake?

Curly: Burned at the stake!

Moe: Why?

Curly: A hot steak is always better than a cold chop.
Revgen is offline   Reply With Quote
Old 14th January 2013, 13:32   #28  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,017
The download link is broken. If you have trouble finding reliable hosting I can put a copy of the scripts on my website.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 14th January 2013, 14:24   #29  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 643
Updated r4.
Added 'dct' argument to SMDegrain to set the equivalent argument of MAnalyse.
Added FFT3dGPU to QTGMC as another Denoiser to NoiseProcess. Since FFT3dGPU doesn't support bt=5, the maximum of NoiseTR is restricted to 1.
Renamed 'DftThreads' of QTGMC to 'DenoiseThreads'. The threads of FFT3DFilter is not set in the original version. Now 'DenoiseThreads' sets the threads of both dfttest and FFT3DFilter.
Added nnedi3_rpow2 function.
Replaced HQDering function with HQDering mod(only 8-bit processing).
Added ivtc_txt60mc function.
Added InterFrame function.
Replaced some filters with the native VapourSynth plugins if possible.
Minor bug fixes.

For ivtc_txt60mc there is a problem. MFlowInter seems to have problem running in VapourSynth when pel=2. So you have to set draft=True in ivtc_txt60mc to let it use pel=1 right now.

For InterFrame there is a bigger problem. It's a pity that SVPflow runs about 10x slower in VapourSynth than it runs in AviSynth. Mostly because of the "Avisynth Compat: requested frame XXX not prefetched, using slow method that may deadlock" issue I guess.

Quote:
Originally Posted by Myrsloik View Post
The download link is broken. If you have trouble finding reliable hosting I can put a copy of the scripts on my website.
Ah...I was just about to update the new version so I deleted the old file.
HolyWu is offline   Reply With Quote
Old 14th January 2013, 14:27   #30  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,017
I'll have a look at the svpflow stuff later. I think I simply needed to add it properly to the avisynth prefetch lists.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 14th January 2013, 15:25   #31  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,410
Quote:
Originally Posted by HolyWu View Post
Updated r4.

For InterFrame there is a bigger problem. It's a pity that SVPflow runs about 10x slower in VapourSynth than it runs in AviSynth. Mostly because of the "Avisynth Compat: requested frame XXX not prefetched, using slow method that may deadlock" issue I guess.
Yep- runs way slower than avs and uses almost 1 core only resources.
kolak is offline   Reply With Quote
Old 14th January 2013, 15:48   #32  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
on nnedi3_rpow2, you should use only std.Transpose() instead of your TurnLeft/Right functions. FlipHorizontal and FlipVertical are useless in this case.

Code:
yexpr = 'x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?'
I suspect that Lut2 is faster than Expr in the case of such a complicated expression.

and, since warpsharp package is not compatible with VS, you can't use UnsharpMask().
__________________
my repositories

Last edited by Chikuzen; 15th January 2013 at 15:20.
Chikuzen is offline   Reply With Quote
Old 15th January 2013, 03:35   #33  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,017
The warpsharp package has been fixed since Chikuzen pointed it out and will work in r19 when that's done.

That's one confusing expression. What does it look like in readable form anyway?
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 15th January 2013, 03:53   #34  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 280
Quote:
Originally Posted by Myrsloik View Post
That's one confusing expression. What does it look like in readable form anyway?
Code:
(((x-128)*(y-128))<0) ? 128 : ((abs(x-128)<abs(y-128)) ? x : y)
Are_ is offline   Reply With Quote
Old 15th January 2013, 09:49   #35  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
a benchmark result

Code:
import vapoursynth as vs
import sys

def expr(x, y):
    if (x - 128) * (y - 128) < 0: return 128
    if abs(x - 128) < abs(y - 128): return x
    return y

def get_lut():
    lut = []
    for y in range(256):
        for x in range(256):
            lut.append(expr(x, y))
    return lut

core = vs.Core(threads=1)
core.std.LoadPlugin('vsrawsource.dll')
c0 = core.raws.Source('720p50_mobcal.y4m')
c0 = core.std.Loop(core.std.ShufflePlanes(c0, 0, vs.GRAY), 4) #2016(504x4) frames
c1 = core.raws.Source('720p50_parkrun.y4m')
c1 = core.std.Loop(core.std.ShufflePlanes(c1, 0, vs.GRAY), 4) #2016(504x4) frames

clip = core.std.Expr([c0, c1], ['x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?'])
#clip = core.std.Lut2([c0, c1], get_lut(), planes=0)

clip.output(sys.stdout)
-------------------------------------------------------------
$ for i in {1..3}; do time python3 benchmarch.py > /dev/null; done

      std.Expr      std.Lut2
1st   0m22.792s     0m17.316s
2nd   0m22.782s     0m16.963s
3rd   0m22.791s     0m17.004s
__________________
my repositories

Last edited by Chikuzen; 15th January 2013 at 09:56.
Chikuzen is offline   Reply With Quote
Old 16th January 2013, 13:02   #36  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 643
The most complicated expression in the script is probably the one used by LSFmod's Smode=5.
Code:
'x y = x x x y - abs 16 / log 1 4 / * exp 16 * 0.75 * x y - dup abs / * x y - dup * 16 dup * 4 + * x y - dup * 4 + 16 dup * * / * 1 48 0 = 0 16 48 / dup * dup * ? + 1 48 0 = 0 x y - abs 48 / dup * dup * ? + / * + ?'
# Infix notation: (x==y) ? x : (x+(((((((abs(x-y)/16)^(1/4))*16)*0.75)*((x-y)/abs(x-y)))*((((x-y)^2)*((16^2)+4))/((((x-y)^2)+4)*(16^2))))*((1+((48==0) ? 0 : ((16/48)^4)))/(1+((48==0) ? 0 : ((abs(x-y)/48)^4))))))
Then the difference is much significant.
Code:
import vapoursynth as vs
import sys

def get_lut():
    lut = []
    for y in range(256):
        for x in range(256):
            if x==y:
                lut.append(x)
            else:
                lut.append(max(min(round(x+(((((((abs(x-y)/16)**(1/4))*16)*0.75)*((x-y)/abs(x-y)))*((((x-y)**2)*((16**2)+4))/((((x-y)**2)+4)*(16**2))))*((1+(0 if 48==0 else (16/48)**4))/(1+(0 if 48==0 else (abs(x-y)/48)**4))))), 255), 0))
    return lut

core = vs.Core()
core.avs.LoadPlugin(r'C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecodeNV.dll')
core.std.LoadPlugin(r'C:\Program Files (x86)\VapourSynth\filters\genericfilters.dll')
clip = core.std.Trim(core.avs.DGSource(r'a 1080p24 video.dgi'), 0, 4999)
blur = core.generic.Blur(clip, planes=0)

clip = core.std.Expr([clip, blur], ['x y = x x x y - abs 16 / log 1 4 / * exp 16 * 0.75 * x y - dup abs / * x y - dup * 16 dup * 4 + * x y - dup * 4 + 16 dup * * / * 1 48 0 = 0 16 48 / dup * dup * ? + 1 48 0 = 0 x y - abs 48 / dup * dup * ? + / * + ?', ''])
#clip = core.std.Lut2([clip, blur], lut=get_lut(), planes=0)

clip.output(sys.stdout)
-------------------------------------------------------------
$ time python test.vpy > /dev/null
std.Expr        std.Lut2
10m10.820s      1m49.963s
HolyWu is offline   Reply With Quote
Old 16th January 2013, 13:21   #37  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,017
I suspect you can speed it up a bit. Not that it really matters when a lut is so much faster...

For example "16 dup *" is a constant expression and there are several more constants recalculated every time. You can also rewrite X ^ (1/4) as sqrt(sqrt(X)) which probably is a lot faster than using exp and log.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 16th January 2013, 13:46   #38  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Code:
'x y = x x x y - abs 16 / log 1 4 / * exp 16 * 0.75 * x y - dup abs / * x y - dup * 16 dup * 4 + * x y - dup * 4 + 16 dup * * / * 1 48 0 = 0 16 48 / dup * dup * ? + 1 48 0 = 0 x y - abs 48 / dup * dup * ? + / * + ?'
# Infix notation: (x==y) ? x : (x+(((((((abs(x-y)/16)^(1/4))*16)*0.75)*((x-y)/abs(x-y)))*((((x-y)^2)*((16^2)+4))/((((x-y)^2)+4)*(16^2))))*((1+((48==0) ? 0 : ((16/48)^4)))/(1+((48==0) ? 0 : ((abs(x-y)/48)^4))))))
is this expression really working ?

also, "... (1+((48==0)) ? 0 : ((16/48))^4)"
why you don't write just "1.012346"
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 16th January 2013, 13:59   #39  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 643
In the original expression some constants are parameters, and are replaced with the function's argument values, for example (1/4) is one of it, so can't simply using sqrt(sqrt(X)) if someone specified (1/3) or so. I replaced all parameters with the default values for convenience here. That's why you wonder why I don't write values which could be calculated beforehand like (1+((48==0) ? 0 : ((16/48)^4))).
HolyWu is offline   Reply With Quote
Old 16th January 2013, 17:25   #40  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
another benchmark (Since HolyWu's script did not work on my box, I rewrote it by myself. )
Code:
import vapoursynth as vs
import sys

core = vs.Core(threads=1)
core.std.LoadPlugin('vsrawsource.dll')
core.std.LoadPlugin('genericfilters.dll')

def func(x, y, strength, szrp, spwr, sdmplo, sdmphi):
    if x == y:
        return x
    diff = x - y
    diff2 = diff * diff
    tmp0 = ((abs(diff)/szrp)**(1 / szrp))*szrp*strength * (diff/abs(diff))
    tmp1 = szrp ** 2 + sdmplo
    tmp2 = szrp ** 2
    tmp3 = 1 + ((szrp/sdmphi) ** 4 if (sdmphi != 0) else 0)
    tmp4 = 1 + (abs(diff)/sdmphi ** 4 if (sdmphi != 0) else 0)
    tmp3 /= tmp4
    ret = x + tmp0 * ((diff2 * tmp1) / ((diff2 + sdmplo) * tmp2)) * tmp3
    return min(255, max(0, int(round(ret))))

def get_lut(strength, szrp, spwr, sdmplo, sdmphi):
    lut = []
    for y in range(256):
        for x in range(256):
            lut.append(func(x, y, strength, szrp, spwr, sdmplo, sdmphi))
    return lut

c0 = core.raws.Source('720p50_mobcal.y4m')
c0 = core.std.ShufflePlanes(c0, 0, vs.GRAY)
c1 = core.generic.Blur(c0)

strength, szrp, spwr, sdmplo, sdmphi = 0.75, 16, 4, 4, 0

expr = ('x y = x x x y - abs {Szrp} / log 1 {Spwr} / * exp {Szrp} * {Str} * x y - '
        'dup abs / * x y - dup * {Szrp} dup * {SdmpLo} + * x y - dup * {SdmpLo} + '
        '{Szrp} dup * * / * 1 {SdmpHi} 0 = 0 {Szrp} {SdmpHi} / dup * dup * ? + 1 '
        '{SdmpHi} 0 = 0 x y - abs {SdmpHi} / dup * dup * ? + / * + ?')
expr = expr.format(Szrp=szrp, Spwr=spwr, Str=strength, SdmpLo=sdmplo, SdmpHi=sdmphi)
clip = core.std.Expr([c0, c1], [expr])
#clip = core.std.Lut2([c0, c1], get_lut(strength, szrp, spwr, sdmplo, sdmphi), planes=0)

clip.output(sys.stdout)
the result is
Quote:
std.Expr 0m20.436s
std.Lut2 0m06.396s
Why was a different result brought?
Probably, HolyWo would use anime as a sample.
Since x and y become equal in many cases in the case of anime, almost part of expression does not evaluate and finishes with the head of them(if x == y then return x).
__________________
my repositories

Last edited by Chikuzen; 17th January 2013 at 06:08.
Chikuzen is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:45.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.