MVTools, Depan, DepanEstimate for VapourSynth - Page 4

Mystery Keeper · 10th December 2014, 11:04

An unofficial build of MVTools test version with "dct" parameter back and working.

Reel.Deel · 23rd December 2014, 22:21

Since Mystery Keeper's build is already a few commits behind any plans to release a current binary?

Mystery Keeper · 24th December 2014, 00:45

I just thought I shouldn't build on every commit and waited to be notified about release. Can build anytime.

jackoneill · 24th December 2014, 11:09

Quote:

Originally Posted by Reel.Deel

Since Mystery Keeper's build is already a few commits behind any plans to release a current binary?

I don't think the newer commits change much for the user.

MonoS · 24th December 2014, 14:53

Any information about 16bit??

jackoneill · 24th December 2014, 16:47

Quote:

Originally Posted by MonoS

Any information about 16bit??

It will happen in a few years.

jackoneill · 2nd January 2015, 14:03

Here is v5.

Code:

* Import speedup from the SVP fork. Affects every filter except Super.
* Add "search_coarse" parameter to Analyse, also from SVP.
* Add FlowFPS and BlockFPS.
* Bring back the "dct" parameter to Analyse and Recalculate (thanks to Mystery Keeper). I hope it sticks this time.
* Fix bug with infinite clips and isb=False in Analyse and Recalculate.

I'm told that FlowFPS is too slow for realtime frame rate conversion (24 to 60 fps). BlockFPS is probably fast enough, but its output is even uglier.

mark0077 · 5th January 2015, 20:33

I currently use SVP dlls with a customised InterFrame script in our HTPC. Is v5 above with the speedups and some of the params from SVP close to being everything that has changed from mvtools to SVP, or are there any plans for the full SVP code to be workable with vapoursynth.

Is it worth taking at this early stage for a live system over avisynth 2.6 MT with SVP dlls in terms of quality and speed?

jackoneill · 6th January 2015, 11:35

Quote:

Originally Posted by mark0077

I currently use SVP dlls with a customised InterFrame script in our HTPC. Is v5 above with the speedups and some of the params from SVP close to being everything that has changed from mvtools to SVP, or are there any plans for the full SVP code to be workable with vapoursynth.

Is it worth taking at this early stage for a live system over avisynth 2.6 MT with SVP dlls in terms of quality and speed?

I believe the speedup and the "search_coarse" parameter were the only changes between 2.5.11.3 and the SVP fork's 2.5.11.9. The SVP's additional filters are not open source, as far as I know.

I have heard that Avisynth MT is unstable and not very efficient in the way it does multithreading.

The quality is the same. For the speed, see the comparisons posted earlier in this thread.

Pat357 · 25th January 2015, 19:00

I can't get Degrain (from libmvtools.dll v5) to work. Sometimes it works when I limit the threads to 1 and use only Degrain1.
With Avisynth MVTools version, I have no problems at all and can use threads=8 and MDegrain3.

What happens is that vspipe.exe always crashes when (trying) to produce an output clip. I get no error code or whatever.
When I use VSEDIT-32b, and do "check script" it gives all OK and provides me the correct properties from the output clip, but when I try to preview, the editor crashes the same way as vspipe.exe

Also using the VSFS plugin, it always creates a large .avi file, but as soon as I try to read it using a player, the file disappears and the player gives a black screen.

Code:

Faulting application name: vspipe.exe, version: 0.0.0.0, time stamp: 0x547394ac
Faulting module name: libmvtools.dll, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x0023126e
Faulting process id: 0x112c
Faulting application start time: 0x01d038ba87e6c605
Faulting application path: c:\Program Files (x86)\VapourSynth\core32\vspipe.exe
Faulting module path: c:\Program Files (x86)\VapourSynth\filters\vapoursynth-mvtools-v5-win32\libmvtools.dll
Report Id: c5d8c719-a4ad-11e4-b6fb-005056c00008

Same for VSedit-32 :

Code:

Faulting application name: vsedit-32bit.exe, version: 3.0.0.0, time stamp: 0x548596cb
Faulting module name: libmvtools.dll, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0xc0000005
Fault offset: 0x0023126e
Faulting process id: 0x9c0
Faulting application start time: 0x01d038d06ee303c6
Faulting application path: K:\programs\VapourSynthEditor-32bit\vsedit-32bit.exe
Faulting module path: c:\Program Files (x86)\VapourSynth\filters\vapoursynth-mvtools-v5-win32\libmvtools.dll
Report Id: f88ead17-a4c3-11e4-b6fb-005056c00008

Fault bucket 813441851, type 17
Event Name: APPCRASH
Response: Not available
Cab Id: 0

Problem signature:
P1: vsedit-32bit.exe
P2: 3.0.0.0
P3: 548596cb
P4: libmvtools.dll
P5: 0.0.0.0
P6: 00000000
P7: c0000005
P8: 0023126e
P9: 
P10: 

Attached files:
F:\TEMP\WERD819.tmp.WERInternalMetadata.xml

These files may be available here:
C:\Users\patrick\AppData\Local\Microsoft\Windows\WER\ReportArchive\AppCrash_vsedit-32bit.exe_86177ff6bffc9c9619a498727ee95caed3faf8e0_172cf356

Analysis symbol: 
Rechecking for solution: 0
Report Id: f88ead17-a4c3-11e4-b6fb-005056c00008
Report Status: 0

Here is my "problem" script :

Code:

import vapoursynth as vs
core = vs.get_core(threads=4, accept_lowercase = True)
core.std.LoadPlugin(path=r"c:\Program Files (x86)\VapourSynth\filters\vapoursynth-mvtools-v5-win32\libmvtools.dll")
core.avs.loadplugin(path=r"k:\programs\Neuron\dgmpgdec158\DGDecode.dll")
ret=core.avs.MPEG2Source(r"k:\film\Ghost Rider - Spirit of Vengeance (2012) HDTVRip\good files demuxed\ghost - 1 - MPEG2, 576p25.d2v")
ret=core.std.Trim(ret, 2000 , 5000)
ret = core.resize.Lanczos(clip=ret, width=720, height=480, format=vs.YUV420P8)
src=ret
c = core.mv.Super(src,pel=2, sharp=1)
bv3 = core.mv.Analyse(c, isb = 1, delta = 3, overlap=4)
bv2 = core.mv.Analyse(c, isb = 1, delta = 2, overlap=4)
bv1 = core.mv.Analyse(c, isb = 1, delta = 1, overlap=4)
fv1 = core.mv.Analyse(c, isb = 0, delta = 1, overlap=4)
fv2 = core.mv.Analyse(c, isb = 0, delta = 2, overlap=4)
fv3 = core.mv.Analyse(c, isb = 0, delta = 3, overlap=4)
# ret = core.mv.Degrain1(clip=src, super=c, mvbw=bv1, mvfw=fv1)
ret = core.mv.Degrain3(clip=src, super=c, mvbw=bv1, mvfw=fv1, mvbw2=bv2, mvfw2=fv2 ,mvbw3=bv3, mvfw3=fv3)
ret.set_output()

Any idea why vspipe and vsedit-32 both crash without any error ?
Also if I do "vspipe --info script.vpy - " , it does not crash and gives me the correct properties from output video, no crashes.

i7-970 3.2Ghz processor (6+6 cores), 24 GB RAM, WIN7pro-x64 , full up to date.
Vapoursynth r25 and r26rc2 both tested (32-bit versions), libmvtools.dll is latest v5, Python v3.4.2 (32-bit)
System is not over-clocked.

Other .vpy files that do not use the libmvtools.dll plugin the run great, so the installation is not the problem, I think.

Also, to exclude other factors as much as possible, this script based on "original MVTools" runs just fine :

Code:

import vapoursynth as vs
# import sys
core = vs.get_core(threads=8, accept_lowercase = True)
# core.std.LoadPlugin(path=r"k:\programs\ffms2-r936c59-avs_vsp\ffms2.dll")
# core.std.LoadPlugin(path=r"c:\Program Files (x86)\VapourSynth\filters\vapoursynth-mvtools-v5-win32\libmvtools.dll")
core.avs.loadplugin(path=r"k:\programs\Neuron\dgmpgdec158\DGDecode.dll")
core.avs.LoadPlugin(path=r"k:\programs\AviSynth 2.5\special filters\mvtools-v2.5.11.3\mvtools2.dll")
# core.std.LoadPlugin(path=r"k:\programs\AviSynth 2.5\special filters\Vapoursynth avisynth filters\vsrawsource.dll")
core.avs.LoadPlugin(path=r"k:\programs\AviSynth 2.5\special filters\masktools-v2.0a48\mt_masktools-26.dll")
core.avs.LoadPlugin(path=r"k:\programs\AviSynth 2.5\special filters\MaskTools-v1.5.8\MaskTools.dll")
# ret=core.ffms2.Source(r"k:\film\Battlefield_3_Fault_Line_Full_Trailer_mov_remux.mkv", threads=1)
ret=core.avs.MPEG2Source(r"k:\film\Ghost Rider - Spirit of Vengeance (2012) HDTVRip\good files demuxed\ghost - 1 - MPEG2, 576p25.d2v")
ret=core.std.Trim(ret, 2000 , 7000)
ret = core.resize.Lanczos(clip=ret, width=720, height=408, format=vs.YUV420P8)
src=ret
c = core.avs.MSuper(src,pel=2, sharp=1)
bv3 = core.avs.MAnalyse(c, isb = 1, delta = 3, overlap=4)
bv2 = core.avs.MAnalyse(c, isb = 1, delta = 2, overlap=4)
bv1 = core.avs.MAnalyse(c, isb = 1, delta = 1, overlap=4)
fv1 = core.avs.MAnalyse(c, isb = 0, delta = 1, overlap=4)
fv2 = core.avs.MAnalyse(c, isb = 0, delta = 2, overlap=4)
fv3 = core.avs.MAnalyse(c, isb = 0, delta = 3, overlap=4)
ret = core.avs.MDegrain3(src,c,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400)
# ret = core.avs.Degrain3(clip=src, super=c, mvbw=bv1, mvfw=fv1, mvbw2=bv2, mvfw2=fv2 ,mvbw3=bv3, mvfw3=fv3 , thsad=400)
# ret = core.avs.MDegrain3(c1=src,c2=c,c3=bv1,c4=fv1,c5=bv2,c6=fv2,thSAD=400)
# diff = core.avs.mt_makediff(src,ret1)
# ret2 = core.avs.mt_adddiff(src,diff)
# ret = core.resize.Lanczos(clip=ret, width=1440, height=720, format=vs.YUV420P8)
# last=ret
# ret = core.std.Trim(ret , 1 , 5001 )
# ret.output(sys.stdout, y4m=True)
# print(core.raws.FormatList())
# print(core.list_functions())
ret.set_output()

I spend already 2 full days to try to find the issue, and honestly I'm still clueless ;
- Testing other files and various formats and containers
- Testing with other plugins to read the files : AVISourse, FFMS v2.20, DGdecNV 2048, DG MPEG2Source 1.58, ...
- Memory consumption was never more then 400 MB.

How can I further debug this ? What tools can I use for this ?

Are_ · 25th January 2015, 20:18

This is strange, maybe related to de 32bit version on Windows.
Did you test 64bit version?

I managed a ~5h long encode making use of d2v+qtgmc+mvtools with no problems (also your snippet does not give me any error).

Mystery Keeper · 25th January 2015, 21:05

24GB RAM? Why in the world are you using 32-bit versions? Use 64-bit. Crash with no error might very well be memory allocation error.

jackoneill · 25th January 2015, 23:17

What if you pass "isse=False" to all the MVTools filters?

Also you can replace DGDecode with d2vsource.

foxyshadis · 25th January 2015, 23:39

Hmm, an access violation doesn't sound like an out of memory, but a stray pointer somewhere. jackoneill, have you tried running the Clang Static Analyzer to check for issues? It's in Fedora with

Code:

yum install llvm-clang-analyzer

jackoneill · 26th January 2015, 13:03

I wouldn't know what to do with the static analyzer.

Does this happen to work better? http://ulozto.net/xZ99sXrQ/vapoursyn...f3bd9-win32-7z

If the DLL above still crashes, retest Degrain1 and Degrain2. In this new DLL either they all crash or they all work.

Pat357 · 26th January 2015, 13:16

Quote:

Originally Posted by jackoneill

What if you pass "isse=False" to all the MVTools filters?

Also you can replace DGDecode with d2vsource.

Thanks , but the "isse=False" didn't help.
Script below gave once the expected output, but then every time had crashed vspipe.exe

Adapted script :

Code:

import vapoursynth as vs
# import sys
core = vs.get_core(threads=2, accept_lowercase = True)
core.std.LoadPlugin(path=r"c:\Program Files (x86)\VapourSynth\filters\vapoursynth-mvtools-v5-win32\libmvtools.dll")
core.std.LoadPlugin(path=r"c:\Program Files (x86)\VapourSynth\filters\d2vsource_beta7\32bit\d2vsource.dll")
ret=core.d2v.Source(input=r"k:\film\Ghost Rider - Spirit of Vengeance (2012) HDTVRip\good files demuxed\ghost - 1 - MPEG2, 576p25.d2v")
# ret=core.std.Trim(ret, 2000 , 5000)
ret = core.resize.Lanczos(clip=ret, width=720, height=480, format=vs.YUV420P8)
src=ret
super = core.mv.Super(src ,pel=2, sharp=1 , isse=False)
# bv3 = core.mv.Analyse(super , blksize=8,  isb=True, delta=3, overlap=4)
bv2 = core.mv.Analyse(super , blksize=8,  isb=True, delta=2, overlap=4)
bv1 = core.mv.Analyse(super , blksize=8,  isb=True, delta=1, overlap=4)
fv1 = core.mv.Analyse(super , blksize=8,  isb= False, delta=1, overlap=4)
fv2 = core.mv.Analyse(super , blksize=8,  isb=False, delta=2 , overlap=4)
# fv3 = core.mv.Analyse(super , blksize=8,  isb=False, delta=3, overlap=4)
# ret = core.mv.Degrain1(clip=src, super=super, mvbw=bv1, mvfw=fv1 , thsad=400)
ret = core.mv.Degrain2(clip=src, super=super, mvbw=bv1, mvfw=fv1, mvbw2=bv2, mvfw2=fv2 , thsad=400 )
# ret = core.mv.Degrain3(clip=src, super=super, mvbw=bv1, mvfw=fv1, mvbw2=bv2, mvfw2=fv2 , mvbw3=bv3, mvfw3=fv3 , thsad=400)
ret.set_output()

Pat357 · 26th January 2015, 13:51

Quote:

Originally Posted by jackoneill

I wouldn't know what to do with the static analyzer.

Does this happen to work better? http://ulozto.net/xZ99sXrQ/vapoursyn...f3bd9-win32-7z

If the DLL above still crashes, retest Degrain1 and Degrain2. In this new DLL either they all crash or they all work.

File has been deleted. Please re-upload it.
Thanks !

Pat357 · 26th January 2015, 14:10

Quote:

Originally Posted by jackoneill

I wouldn't know what to do with the static analyzer.

Does this happen to work better? http://ulozto.net/xZ99sXrQ/vapoursyn...f3bd9-win32-7z

If the DLL above still crashes, retest Degrain1 and Degrain2. In this new DLL either they all crash or they all work.

Got the file. Thanks for that.

Now Degrain2 sometimes works with "threads=1/2"', Degrain3 only with threads=1. Still not stable though : 4 out of 5 attempts still crash vspipe.exe.

I really doubt that memory is an issue here : using the original MVTools, I can use MDegrain3 with 8 threads on the same input video....

Do you need more crash reports ?

jackoneill · 26th January 2015, 14:16

I was able to get it to crash in a 32 bit Windows XP, but passing isse=False to Degrain2 and Degrain3 makes them work. Are you sure this doesn't help you?

Pat357 · 26th January 2015, 14:32

Quote:

Originally Posted by jackoneill

I was able to get it to crash in a 32 bit Windows XP, but passing isse=False to Degrain2 and Degrain3 makes them work. Are you sure this doesn't help you?

Thank you very much !
The first time you mentioned isse=False, I've added it only to Super().
Now I've added it to Degrain2 and Degrain3 and with success !

I'm now running Degrain3 with "threads=4" and still no crashes.

BTW I'm running Win7Pro-64 bit to be able to access my 24GB RAM. Non-Pro Win7-64 can only address 16 GB RAM IIRC.

Does this "isse=False" does what I think it does ? Not using SSE instructions ?
Would this make the script slower ?

10th December 2014, 11:04	#61 \| Link
Mystery Keeper Beyond Kawaii Join Date: Feb 2008 Location: Russia Posts: 724	An unofficial build of MVTools test version with "dct" parameter back and working. __________________ ...desu!

24th December 2014, 00:45	#63 \| Link
Mystery Keeper Beyond Kawaii Join Date: Feb 2008 Location: Russia Posts: 724	I just thought I shouldn't build on every commit and waited to be notified about release. Can build anytime. __________________ ...desu!

2nd January 2015, 14:03	#67 \| Link
jackoneill unsigned int Join Date: Oct 2012 Location: 🇪🇺 Posts: 760	Here is v5. Code: * Import speedup from the SVP fork. Affects every filter except Super. * Add "search_coarse" parameter to Analyse, also from SVP. * Add FlowFPS and BlockFPS. * Bring back the "dct" parameter to Analyse and Recalculate (thanks to Mystery Keeper). I hope it sticks this time. * Fix bug with infinite clips and isb=False in Analyse and Recalculate. I'm told that FlowFPS is too slow for realtime frame rate conversion (24 to 60 fps). BlockFPS is probably fast enough, but its output is even uglier. __________________ Buy me a "coffee" and/or hire me to write code!

25th January 2015, 21:05	#72 \| Link
Mystery Keeper Beyond Kawaii Join Date: Feb 2008 Location: Russia Posts: 724	24GB RAM? Why in the world are you using 32-bit versions? Use 64-bit. Crash with no error might very well be memory allocation error. __________________ ...desu!

25th January 2015, 23:17	#73 \| Link
jackoneill unsigned int Join Date: Oct 2012 Location: 🇪🇺 Posts: 760	What if you pass "isse=False" to all the MVTools filters? Also you can replace DGDecode with d2vsource. __________________ Buy me a "coffee" and/or hire me to write code!

23rd December 2014, 22:21	#62 \| Link
Reel.Deel Registered User Join Date: Mar 2012 Location: Texas Posts: 1,666	Since Mystery Keeper's build is already a few commits behind any plans to release a current binary?

24th December 2014, 14:53	#65 \| Link
MonoS Registered User Join Date: Aug 2012 Posts: 203	Any information about 16bit??

5th January 2015, 20:33	#68 \| Link
mark0077 Registered User Join Date: Apr 2008 Posts: 1,106	I currently use SVP dlls with a customised InterFrame script in our HTPC. Is v5 above with the speedups and some of the params from SVP close to being everything that has changed from mvtools to SVP, or are there any plans for the full SVP code to be workable with vapoursynth. Is it worth taking at this early stage for a live system over avisynth 2.6 MT with SVP dlls in terms of quality and speed?

25th January 2015, 20:18	#71 \| Link
Are_ Registered User Join Date: Jun 2012 Location: Ibiza, Spain Posts: 321	This is strange, maybe related to de 32bit version on Windows. Did you test 64bit version? I managed a ~5h long encode making use of d2v+qtgmc+mvtools with no problems (also your snippet does not give me any error).

25th January 2015, 23:39	#74 \| Link
foxyshadis Angel of Night Join Date: Nov 2004 Location: Tangled in the silks Posts: 9,559	Hmm, an access violation doesn't sound like an out of memory, but a stray pointer somewhere. jackoneill, have you tried running the Clang Static Analyzer to check for issues? It's in Fedora with Code: yum install llvm-clang-analyzer

26th January 2015, 13:03	#75 \| Link
jackoneill unsigned int Join Date: Oct 2012 Location: 🇪🇺 Posts: 760	I wouldn't know what to do with the static analyzer. Does this happen to work better? http://ulozto.net/xZ99sXrQ/vapoursyn...f3bd9-win32-7z If the DLL above still crashes, retest Degrain1 and Degrain2. In this new DLL either they all crash or they all work. __________________ Buy me a "coffee" and/or hire me to write code! Last edited by jackoneill; 26th January 2015 at 13:28.

26th January 2015, 14:16	#79 \| Link
jackoneill unsigned int Join Date: Oct 2012 Location: 🇪🇺 Posts: 760	I was able to get it to crash in a 32 bit Windows XP, but passing isse=False to Degrain2 and Degrain3 makes them work. Are you sure this doesn't help you? __________________ Buy me a "coffee" and/or hire me to write code!