HAvsFunc [Archive] - Page 8

tuanden0

12th May 2017, 04:38

Can someone help me fix that error?

Failed to evaluate the script:
Python exception: No attribute with the name scd exists. Did you mistype a plugin namespace?

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 1821, in vapoursynth.vpy_evaluateScript (src\cython\vapoursynth.c:36464)
File "E:/Download/test.vpy", line 9, in
video = haf.LSFmod(video)
File "C:\Users\Home\AppData\Local\Programs\Python\Python36\lib\site-packages\havsfunc.py", line 4072, in LSFmod
diff = core.std.Expr([diff, TemporalSoften(diff, 1, 255 File "C:\Users\Home\AppData\Local\Programs\Python\Python36\lib\site-packages\havsfunc.py", line 4312, in TemporalSoften
clip = set_scenechange(clip, scenechange)
File "C:\Users\Home\AppData\Local\Programs\Python\Python36\lib\site-packages\havsfunc.py", line 4344, in set_scenechange
sc = core.scd.Detect(sc, thresh)
File "src\cython\vapoursynth.pyx", line 1390, in vapoursynth.Core.__getattr__ (src\cython\vapoursynth.c:29151)
AttributeError: No attribute with the name scd exists. Did you mistype a plugin namespace?

I fixed, there were missing some file in here:
https://forum.doom9.org/showthread.php?t=166769

HolyWu

6th June 2017, 14:05

Update r26.

srestore: Only do format conversion when necessary.
Deprecate the use of focus2/scd plugin and use misc plugin instead.
MinBlur: Remove workaround for CTMF in r=2 when input is 16 bits.

HolyWu

23rd October 2017, 15:16

Update r27.

Add TemporalDegrain and aaf functions. (Hinterwaeldlers)
HQDeringmod: Keep up with the changes in std.Sobel.
santiag, QTGMC: Use the optimized eedi3 filter instead of the bundled one.
daa, santiag, QTGMC: Add opencl parameter. The OpenCL version of eedi3/nnedi3 will be used when set to true. By default it's set to false and uses the CPU version like before.

KingLir

25th November 2017, 20:26

When adding the new opencl parameter to my working QTGMC line (below) I am getting the following error. Any ideas what I need to do ?

clip = haf.QTGMC(clip, Preset='Placebo', MatchPreset='Placebo', MatchPreset2='Placebo', EdiQual=2, SourceMatch=3, TR1=2, TR2=3, Lossless=2, EZKeepGrain=0.0, EZDenoise=0.4, Sharpness=0.5, TFF=True, opencl=True)

File "/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/havsfunc.py", line 1114, in QTGMC
edi1 = QTGMC_Interpolate(ediInput, InputType, EdiMode, NNSize, NNeurons, EdiQual, EdiMaxD, bobbed, ChromaEdi, TFF, opencl)
File "/usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/havsfunc.py", line 1401, in QTGMC_Interpolate
interp = myNNEDI3(Input, field=field, planes=planes, nsize=NNSize, nns=NNeurons, qual=EdiQual)
File "src/cython/vapoursynth.pyx", line 1722, in vapoursynth.Function.__call__
vapoursynth.Error: NNEDI3CL: Invalid Program

HolyWu

26th November 2017, 05:42

When adding the new opencl parameter to my working QTGMC line (below) I am getting the following error. Any ideas what I need to do ?

vapoursynth.Error: NNEDI3CL: Invalid Program

Can you confirm that your GPU supports OpenCL 1.2? If so, can you run the latest version of KNLMeansCL as well?

KingLir

26th November 2017, 11:29

Can you confirm that your GPU supports OpenCL 1.2? If so, can you run the latest version of KNLMeansCL as well?

The GPU does support it. I got iMac (Retina 5K, 27-inch, Late 2015) with AMD Radeon R9 M395X (4GB VRAM) with OpenCL 1.2 support as shown here:
https://support.apple.com/en-il/HT202823

Latest macOS (and therefore latest drivers and OpenCL Apple framework Version 2.8.11) installed.

Actually latest KNLMeansCL version works (but with some artifacts - precision issue). KNLMeansCL log also state that i got OpenCL 1.2 .

HolyWu

26th November 2017, 20:00

The GPU does support it. I got iMac (Retina 5K, 27-inch, Late 2015) with AMD Radeon R9 M395X (4GB VRAM) with OpenCL 1.2 support as shown here:
https://support.apple.com/en-il/HT202823

Please recompile the source from debug branch (https://github.com/HomeOfVapourSynthEvolution/VapourSynth-NNEDI3CL/tree/debug) and post the result to somewhere like pastebin.

KingLir

26th November 2017, 22:48

Please recompile the source from debug branch (https://github.com/HomeOfVapourSynthEvolution/VapourSynth-NNEDI3CL/tree/debug) and post the result to somewhere like pastebin.

Thanks. Here:
https://pastebin.com/8PMj2ZEm

HolyWu

27th November 2017, 08:14

Thanks. Here:
https://pastebin.com/8PMj2ZEm

Please try the latest commit in debug branch again, thanks.

KingLir

27th November 2017, 13:07

Please try the latest commit in debug branch again, thanks.

Thanks ! It works now.

1. The output is different as without OpenCL use - smaller file size. I also used binary compare and found 1975 changes all across the file - not limited to specific part of the file.
These are 2 new encodes I just did with the only different is the added " opencl=True ".
Although playing both looks the same.
Any idea why ?

2. Will you be able to set the configure file to use " -framework OpenCL " instead of " -lOpenCL " when creating the makefile on macOS ? I am doing it manually each time.

3. The performance of QTGMC CL (the specific line I use is on my previous post) is a bit slower comparing to without the opencl=True .
I guess my CPU is just better for this than my GPU.
EDIT: WRONG - SEE MY TESTING BELOW

4. KLMeansCL still produce artifacts. Got any ideas I can pass on to Khanattila ?

BTW- Does this OpenCL feature for QTGMC is only available for us VapourSynth users or AviSynth users have also access to something similar ?

HolyWu

27th November 2017, 21:10

Thanks ! It works now.

1. The output is different as without OpenCL use - smaller file size. I also used binary compare and found 1975 changes all across the file - not limited to specific part of the file.
These are 2 new encodes I just did with the only different is the added " opencl=True ".
Although playing both looks the same.

2. Will you be able to set the configure file to use " -framework OpenCL " instead of " -lOpenCL " when creating the makefile on macOS ? I am doing it manually each time.

3. The performance of QTGMC CL (the specific line I use is on my previous post) is a bit slower comparing to without the opencl=True .
I guess my CPU is just better for this than my GPU.

4. KLMeansCL still produce artifacts. Got any ideas I can pass on to Khanattila ?

BTW- Does this OpenCL feature for QTGMC is only available for us VapourSynth users or AviSynth users have also access to something similar ?

1. There is no point in comparing binary output. Both the CPU version and the OpenCL version have their own technique of trading quality off against speed, hence you never get bit-exact output.

2. Will fix.

3. Probably. And I just push a commit to use znedi3 instead of the original nnedi3 now.

4. Either https://forum.doom9.org/showthread.php?t=171379 or https://github.com/Khanattila/KNLMeansCL/issues.

The AviSynth's nnedi3ocl only has nsize=0 implemented and has no prescreener so nobody really cares to use it in QTGMC.

KingLir

27th November 2017, 23:15

1. There is no point in comparing binary output. Both the CPU version and the OpenCL version have their own technique of trading quality off against speed, hence you never get bit-exact output.

I see. So how can I know which gives better image quality ?

2. Will fix.

Thanks!

3. Probably. And I just push a commit to use znedi3 instead of the original nnedi3 now.

Nice. Can you help me build a macOS version ?
This (https://github.com/sekrit-twc/znedi3) doesn't have autogen/configure and I am getting the following from:

A. simple "make" :
c++ -c -o vsznedi3/vsznedi3.o vsznedi3/vsznedi3.cpp
vsznedi3/vsznedi3.cpp:8:10: fatal error: 'znedi3.h' file not found

B. simple gcc command:
https://pastebin.com/mmitrMCV

4. Either https://forum.doom9.org/showthread.php?t=171379

This link just show the OP of KMLMeans.... did you mean something else ?

The AviSynth's nnedi3ocl only has nsize=0 implemented and has no prescreener so nobody really cares to use it in QTGMC.

Good to know :)

DJATOM

27th November 2017, 23:57

KingLir
Try to google for those warnings, you might find a solution. At least for C++11 warnings CXXFLAGS=-g -std=c++11 -Wall -pedantic should help.
About "'znedi3.h' file not found": compiler can't find your includes. Set them with "-I./path/to/include/dir". You can add a couple of -I options if needed.

DJATOM

28th November 2017, 00:47

https://github.com/thebombzen/znedi3 - repo with Makefile.

HolyWu

28th November 2017, 06:43

I see. So how can I know which gives better image quality ?

By eyes. For instance, you probably won't notice significant differences using Interleave([nnedi3(clip, field=1, exp=0), nnedi3(clip, field=1, exp=2)]), even zoom in at 300%. But if you use MakeDiff(nnedi3(clip, field=1, exp=0), nnedi3(clip, field=1, exp=2)).Levels(min_in=127, max_in=129, min_out=0, max_out=255) you can easily see there are a pile of pixel differences. However do you or anyone bother to turn off those speed-ups which sacrifice precision? I guess no. Then why bother now?

Nice. Can you help me build a macOS version ?
This (https://github.com/sekrit-twc/znedi3) doesn't have autogen/configure and I am getting the following from:

A. simple "make" :
c++ -c -o vsznedi3/vsznedi3.o vsznedi3/vsznedi3.cpp
vsznedi3/vsznedi3.cpp:8:10: fatal error: 'znedi3.h' file not found

The makefile in the official repo works fine for me: https://pastebin.com/un1TvN7b

This link just show the OP of KMLMeans.... did you mean something else ?

I don't get it. Don't you want to report something to Khanattila? Then don't you need to post in the relevant thread or file an issue in the relevant github repo?

KingLir

28th November 2017, 16:36

By eyes. For instance, you probably won't notice significant differences using Interleave([nnedi3(clip, field=1, exp=0), nnedi3(clip, field=1, exp=2)]), even zoom in at 300%. But if you use MakeDiff(nnedi3(clip, field=1, exp=0), nnedi3(clip, field=1, exp=2)).Levels(min_in=127, max_in=129, min_out=0, max_out=255) you can easily see there are a pile of pixel differences. However do you or anyone bother to turn off those speed-ups which sacrifice precision? I guess no. Then why bother now?

I see your point :)
Just for my knowledge, how does one turn off those speed-ups in QTGMC ?

The makefile in the official repo works fine for me: https://pastebin.com/un1TvN7b

Thanks, I don't know why I got the errors before. Anyway, I update the makefile for .dylib instead of .o creation and seems to work for macOS.

I don't get it. Don't you want to report something to Khanattila? Then don't you need to post in the relevant thread or file an issue in the relevant github repo?

I actually post in the past and Khanattila don't have a lead for the precision issues (that he estimate are the cause for the artifacts). I asked if you got ideas after your quickly managed to fix QTGMC CL for me....

KingLir

28th November 2017, 17:19

Something else... I did a more correct testing (below) and found out that:
A. OpenCL is significantly faster than nnedi3 on my System, Although I got a good CPU and only laptop-grade GPU ( details below ).
B. Something is wrong with znedi3 (at least for me). It's super slow. How can I help to debug this ?

My System:
CPU- Intel Core i7 Skylake 4.0 GHz quad-core (Turbo Boost up to 4.2 GHz) 8 MB shared L3 cache (i7-6700K)
GPU - AMD Radeon R9 M395X 4 GB ("Laptop-grade" GPU)
RAM- 24 GB 1867 MHz DDR3

Output 1236 frames:

opencl (HAvsFunc latest github) - in 49.95 seconds (24.75 fps)
nnedi3 (HAvsFunc r27) - in 74.88 seconds (16.51 fps)
znedi3 (HAvsFunc latest github) - in 362.15 seconds (3.41 fps)

HolyWu

30th November 2017, 19:51

I see your point :)
Just for my knowledge, how does one turn off those speed-ups in QTGMC ?

At the moment you can't 'coz the relevant parameters haven't been exposed for years, unless you manually edit the script. I'll probably add them in a few days though.

B. Something is wrong with znedi3 (at least for me). It's super slow. How can I help to debug this ?

You should test by the filter itself first, not by a complex function. The vpy should only contain a source filter followed by nnedi3/znedi3 and nothing else. If the result is still the same, fire an issue on the GitHub repo and provide necessary information.

KingLir

30th November 2017, 21:32

At the moment you can't 'coz the relevant parameters haven't been exposed for years, unless you manually edit the script. I'll probably add them in a few days though.

Cool. Thanks !

You should test by the filter itself first, not by a complex function. The vpy should only contain a source filter followed by nnedi3/znedi3 and nothing else. If the result is still the same, fire an issue on the GitHub repo and provide necessary information.

Sorry if I making hard time for you here. Just want to help out.
But may I ask you post the simplest vpy ffms2-based you recommend for testing ? since simple is a relative term :)
Also, what exactly should be included in "necessary information" ?

KingLir

30th November 2017, 22:46

You probably did not compile it with x86 support ("make X86=1").

Thank you. I just pulled the latest push from today and built a new dylib with make X86=1. Still the same performance:

encoded 1236 frames in 351.54s (3.52 fps)

KingLir

30th November 2017, 22:58

BTW- this is the rough patch I did (.diff) to add macOS build support:
https://pastebin.com/FUaSBPYZ

KingLir

1st December 2017, 09:08

Did you "make clean"? Do you see x86 object files in the Makefile output? What if you pass "x_cpu='avx2'"?

Thanks! I did forgot the cleaning. Used for it being done automatically after build. Maybe you should also add that to the makefile (in addition to macOS support I sent in my .diff) ?

So it seems fixed now. BTW I did another clean and then make x_cpu='avx2' - it was back to being super slow.

Still OpenCL is significantly faster on my system. And once I test on faster "desktop-class" GPU I am assuming it will be much faster.

znedi3:
Output 1236 frames in 62.87 seconds (19.66 fps)

OpenCL:
Output 1236 frames in 50.26 seconds (24.59 fps)

KingLir

1st December 2017, 22:08

Maybe your memory is too slow to support higher performance. 24 GB sounds like it is running in single-channel mode.

Doesn't seem like the issue:

https://preview.ibb.co/cTXofG/Screen_Shot_2017_12_01_at_22_47_50.png

It does make sense that znedi3 will be faster. Any ideas for me to try ?

KingLir

2nd December 2017, 09:19

Based on the results in the other thread, znedi3 should be 50% faster than nnedi3 per-clock on Haswell and Skylake. However, running nnedi3 50% faster is not the same as running QTGMC 50% faster.

By the way, the "laptop-grade" R9 M395X has 3 TFLOPS of compute, so it is not surprising for it to be faster than the i7-6700K's 512 GFLOPS. znedi3 only begins outperforming GPUs on HEDT, particularly Skylake-X (i7-7800K has 1.5 TFLOPS).

Thank you for the explanation ! So it seems everything is normal on my system.

masterkivat

3rd December 2017, 21:27

Can someone help me posting a usage example of LogoNR from HAvsFunc? :thanks:

lansing

4th December 2017, 06:58

Can someone help me posting a usage example of LogoNR from HAvsFunc? :thanks:
You'll need to get the delogo package here:
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-DeLogo

And then look for your lgd logo file from this logo collection (https://down.7086.in/lgd/), they attached all the (l,t,r,b) settings already.

Original from nmm:
https://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=1244

If you can't find your logo, then you'll need to make it yourself, in which I don't know how.

import vapoursynth as vs
import havsfunc as haf

src = core.avisource.AVISource(r"abc.avi")
dlg = core.delogo.EraseLogo(src, r"logo.lgd")

clip = haf.logoNR(dlg, src, l=1280, t=40, r=-40, b=-960)

HolyWu

4th December 2017, 07:40

Unlike the fashion of AviSynth's Crop, the values of r and b need to be positive, just like std.Crop.

masterkivat

4th December 2017, 08:06

You'll need to get the delogo package here:
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-DeLogo

And then look for your lgd logo file from this logo collection (https://down.7086.in/lgd/), they attached all the (l,t,r,b) settings already.

Original from nmm:
https://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=1244

If you can't find your logo, then you'll need to make it yourself, in which I don't know how.

import vapoursynth as vs
import havsfunc as haf

src = core.avisource.AVISource(r"abc.avi")
dlg = core.delogo.EraseLogo(src, r"logo.lgd")

clip = haf.logoNR(dlg, src, l=1280, t=40, r=-40, b=-960)
Thank you, lansing!
To be honest, I'm familiar with logoNR from avisynth, but I'm start learning how to vapoursynth and sometimes I get lost with some strings and hierarchy, but I'll get there. Thanks again! :thanks:

STJAM

11th February 2018, 03:02

HolyWu

11th February 2018, 04:19

I would like to use both havsfunc.py - Stab and FFT3Dfilter in the same script but
when I load plugin DePan which require libfftw3f-3.dll to be in the search path I receive the following.

vsedit: vsedit.exe - Entry Point Not Found
The proceedure entry point fftwf_make_planner_thread_safe
could not be located in the dynamic link library
C:\VapourSynth-FFT3DFilter_test3\x64\fft3dfilter.dll.

I was experiencing the same problem using DFFtest with FFT3DFilter until I realized I was using an older release of DFTest. Which makes me think an update for Depan is in order and it is not a problem with fft3dfilter.
The Stab in havsfunc.py is using as far as I can tell http://github.com/HomeOfVapourSynthEvolution/VapourSynth-DePan/tree/master/DePan
instead of http://github.com/dubhater/vapoursynth-mvtools/releases. If Stab could be written to use the mvtools release.

Neither DePan nor DFTTest uses fftwf_make_planner_thread_safe, but FFT3DFilter. I have a feeling that there is an old version of libfftw3f-3 in your system got loaded, which doesn't have fftwf_make_planner_thread_safe.

I'd like to use the depan-related filters in mvtools long time ago. But DepanEstimate in mvtools lacks the range parameter for unknown reason, which is set to 1 by default in Stab. I have quite few experence in using Stab. If the result is still fine when range=0 then I can simply change it in the next release.

HolyWu

10th March 2018, 16:40

Update r28.

Replace nnedi3 with more optimized znedi3.
daa, santiag, QTGMC: Expose more parameters of nnedi3/eedi3.
ivtc_txt60mc: Add 'opencl' parameter and fix framerate mismatch error.
logoNR: Change the default values for KNLMeansCL.
Deblock_QED: Use renewed DCTFilter.
MinBlur: Performance improvement (Z4ST1N).
QTGMC: Fix incorrect noiseCentre when Denoiser='KNLMeansCL'.
QTGMC: Keep up with the changes in FFT3DFilter.
Add MCTemporalDenoise function (no post-processor).

Selur

10th March 2018, 17:50

Thanks! :)

poisondeathray

10th March 2018, 17:51

Add MCTemporalDenoise function (no post-processor).

Thanks :)

lansing

10th March 2018, 18:56

Wow we finally got the best denoiser in vapoursynth, thanks

Selur

11th March 2018, 10:41

For me using settings="very high":
# Imports
import os
import sys
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("G:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'G:/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/vsfft3dfilter.dll")
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/Support/DCTFilter.dll")
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/DeblockFilter/Deblock/Deblock.dll")
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import havsfunc
# Loading F:\TestClips&Co\SILVERFANG_multiple_subs.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/TESTCL~1/SILVER~1.MKV", format="YUV420P8", cache=0)
# denoising using MCTemporalDenoise
clip = havsfunc.MCTemporalDenoise(i=clip, settings="very high")
# Output
clip.set_output())
Simply crashes vsedit and vspipe. All other 'settings' values work fine.
Using 'clip = havsfunc.MCTemporalDenoise(i=clip, settings="very high", twopass=False)' doesn't help either, so it's not the twopass setting.

Cu Selur

ChaosKing

11th March 2018, 10:56

No problems here with very high settings. Have you updated to latest fft3dfilter? I had to download Test3 https://forum.doom9.org/showthread.php?t=175199

Edit:
My CPU is a Ryzen 1700

Edit2:
I use a sliglty newer libfftw3f-3.dll version (not the one bundled with Test3)

Selur

11th March 2018, 10:57

Okay, will try to investigate some further may be it's one of the dlls I use,..
-> yes, seems like I used an old fft3dfilter version which was buggy :)

ChaosKing

11th March 2018, 11:24

@HolyWu could you add an float opacity parameter to Overlay() like in avisynth?

hydra3333

11th March 2018, 11:59

znedi3
Is that https://github.com/sekrit-twc/znedi3/releases ?
I'm not sure if it's 32bit or 64bit (no separate folders provided).

Selur

11th March 2018, 12:03

@hydra3333: Works here, so it should be 64bit, since I use 64bit Vapoursynth. :)

Cu Selur

HolyWu

11th March 2018, 14:27

@HolyWu could you add an float opacity parameter to Overlay() like in avisynth?

Added.

ChaosKing

11th March 2018, 18:05

Thank you, works good!

yoon

19th March 2018, 21:10

Is there any way to limit qtgmc's memory usage? It completely freezes my Mac.

My script is really simple:

import vapoursynth as vs
import havsfunc as haf

core = vs.get_core()

video = core.lsmas.LWLibavSource(source=path)

video = haf.QTGMC(video, Preset="Slower", TFF=True)

video = core.fmtc.resample(video, 629,354, kernel = "bilinear", invks = True, css="444")

video = core.std.CropRel(video, left=180, top=3, right=181, bottom=1)

video = core.fmtc.resample(video, css="420")
video = core.fmtc.bitdepth(video, bits=8)

Pipe:

vspipe --y4m test.vpy - | ffmpeg -f yuv4mpegpipe -colorspace bt709 -i - -vcodec rawvideo -pix_fmt bgr24 -y output.mov

It never goes past the tenth frame : (

My Mac specs:

OS: High Sierra
Intel Core i5 2.4GHz
Intel HD Graphics + Nvidia GT 330M
4GB RAM

HolyWu

20th March 2018, 04:02

Is there any way to limit qtgmc's memory usage? It completely freezes my Mac.

You can use max_cache_size (http://www.vapoursynth.com/doc/pythonreference.html#Core.max_cache_size) to set the upper framebuffer cache size after obtaining the core instance, such as core.max_cache_size = 2048.