Vapoursynth

[Myrsloik]

Originally Posted by Mystery Keeper

Right, sorry. Restarted the encode with memory limit changed to 2500. 9K frames so far. Memory usage isn't going over 840MB, but IS slowly rising.
Update: 100000 frames. Still the same fluctuating bitrate. Memory usage still not going over 840MB.

You are running it from a command prompt, right? That way it will usually at least print a fatal error message to stderr. It could be that it simply went over the limit very shortly. I would never try to run it with a memory limit over 2GB since that's only the limit for when memory begins to get reclaimed. Combined with allocations inside filters you're still dangerously close to 3GB if you set it to 2.5GB.

[Mystery Keeper]

Nope. I'm simply encoding it in VirtualDub GUI. Right now it is over 145000 frames, and is still using around 830MB. No idea why it would crash back then.
Update. It has suddenly and silently crashed again.

[Myrsloik]

Originally Posted by Mystery Keeper

Nope. I'm simply encoding it in VirtualDub GUI. Right now it is over 145000 frames, and is still using around 830MB. No idea why it would crash back then.
Update. It has suddenly and silently crashed again.

Try doing the encoding with vspipe next time. That way you'll most likely see the fatal error printed. I'm going to try your script now myself and see what happens...
What's the resolution of your source?

[Mystery Keeper]

Quite ordinary 720x480 NTSC DVD.

RemovePort is now done being ported and converted to intrinsics. If you want to test it out a binary is here. I have not really tested it much so far so please let me know of any issues.

A simple Vapoursynth script to test would look like the following if your clip is not greyscale:

clip = core.ffms2.Source(src)
cleansed = core.rgvs.Clense(clip)
sbegin = core.rgvs.ForwardClense(clip)
send = core.rgvs.BackwardClense(clip)
scenechange = core.vsrd.SCSelect(clip, sbegin, send, cleansed)
alt = core.rgvs.Repair(scenechange, clip, mode=[16,16,1])
restore = core.rgvs.Repair(cleansed, clip, mode=[16,16,1])
corrected = core.vsrd.RestoreMotionBlocks(cleansed, restore, neighbour=clip, alternative=alt, gmthreshold=70, dist=1, dmode=2, noise=10, noisy=12, grey=0)
clip = core.rgvs.RemoveGrain(corrected, mode=[17,17,1])
clip.set_output()

If it is greyscale change the second value in the mode array of ints to -1. A more fancy function will be knocked up later.

Works with only YUV420P8 and YUV422P8 right now.

[easyfab]

I try vapoursynth for the first time. It's a little bit harder than avisynth.
But I succeed to run a script with qtgmc() thanks to aegisofrime script example http://forum.doom9.org/showthread.ph...99#post1649699

although the cpu usage is 100% ( compare to ~50% with my avs script ) the speed is slower for x264 encoding ???
Are some plugins less optimized yet ?

Another question:
Is there a special way to load yadif.dll ? because it doesn't work for me

core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll')
core.avs.Load_Stdcall_Plugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll') ?

And if someone can give me a script example for yadifmod with edeint=nnedi3 It would be nice.

Vapoursynth look promising for the futur, I will keep an eye on it .

[Myrsloik]

Originally Posted by easyfab

I try vapoursynth for the first time. It's a little bit harder than avisynth.
But I succeed to run a script with qtgmc() thanks to aegisofrime script example http://forum.doom9.org/showthread.ph...99#post1649699

although the cpu usage is 100% ( compare to ~50% with my avs script ) the speed is slower for x264 encoding ???
Are some plugins less optimized yet ?

Another question:
Is there a special way to load yadif.dll ? because it doesn't work for me

core.avs.LoadPlugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll')
core.avs.Load_Stdcall_Plugin(path=r'C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll') ?

And if someone can give me a script example for yadifmod with edeint=nnedi3 It would be nice.

Vapoursynth look promising for the futur, I will keep an eye on it .

Avisynth C plugins can't be loaded. Only normal 2.5 API ones work. Exactly what were your QTGMC and x264 settings when testing it?

QTGMC is known to be a bit slower than when run in avisynth-mt at its best but the difference should be around 15% or so at most.

[Myrsloik]

Originally Posted by Mystery Keeper

Quite ordinary 720x480 NTSC DVD.

Did you overclock your computer or look at the core temperature at all when running the script? Just curious.

[Mystery Keeper]

Originally Posted by Myrsloik

Did you overclock your computer or look at the core temperature at all when running the script? Just curious.

Nope. Water cooled and not overclocked.

[Myrsloik]

Originally Posted by Mystery Keeper

Nope. Water cooled and not overclocked.

Running the script with a blankclip source now. Will see if it can pass 500000 frames in a couple of hours. Days like these I wish I'd gone with a 3770k cpu...

[Mystery Keeper]

That's why I'm saving for two 12 cores Xeons ^_^'

[easyfab]

Originally Posted by Myrsloik

Avisynth C plugins can't be loaded. Only normal 2.5 API ones work. Exactly what were your QTGMC and x264 settings when testing it?

QTGMC is known to be a bit slower than when run in avisynth-mt at its best but the difference should be around 15% or so at most.

For vapoursynth ( 100% cpu usage )
clip = haf.QTGMC(clip, Preset='slow',TFF=True)
clip = clip[::2]

and for avisynth (~50-60% cpu usage )

setmtmode(2)
QTGMC(preset="slow", EdiThreads=4).selecteven()

x264 --preset slower

And speed is about 20% less with vapoursynth

[mastrboy]

Originally Posted by Myrsloik

Running the script with a blankclip source now. Will see if it can pass 500000 frames in a couple of hours. Days like these I wish I'd gone with a 3770k cpu...

Now I'm curious, what did you get instead?

[Myrsloik]

Originally Posted by mastrboy

Now I'm curious, what did you get instead?

The 3570k, because at the time I hadn't decided to work on very multithreaded things like VapourSynth. Maybe Intel will give me something better if I ask nicely...

Anyway, next investment will be a very fast graphics card that doesn't sound like a vacuum cleaner.

If someone happens to have a computer with lots of cores I could use remotely to test VapourSynth on that'd be very useful. It's hard for me to test how well things scale on lots of cores with only 4 here.

[Myrsloik]

Originally Posted by Mystery Keeper

Quite ordinary 720x480 NTSC DVD.

506k frames later and it still works. I don't think this method of testing will yield anything. I'll add logging of fatal errors and then let you try that build.

[Reel.Deel]

Originally Posted by handaimaoh

RemovePort is now done being ported and converted to intrinsics. If you want to test it out a binary is here. I have not really tested it much so far so please let me know of any issues.

When I try to load RemoveDirt I get this error:

---------------------------
VirtualDub Error
---------------------------
Python exception: 'Failed to load C:\\RemoveDirt.dll'
Traceback (most recent call last):
File "vapoursynth.pyx", line 1082, in vapoursynth.vpy_evaluateScript (src\cython\vapoursynth.c:16736)
File "C:\test.vpy", line 4, in <module>
core.std.LoadPlugin(path=r'C:\RemoveDirt.dll')
File "vapoursynth.pyx", line 1005, in vapoursynth.Function.__call__ (src\cython\vapoursynth.c:15793)
vapoursynth.Error: 'Failed to load C:\\RemoveDirt.dll'

Also, since clense is now included with RemoveGrainVS shouldn't the script look something like this instead?

clip = core.ffms2.Source(src)
cleansed = core.rgvs.Clense(clip)
sbegin = core.rgvs.ForwardClense(clip)
send = core.rgvs.BackwardClense(clip)
scenechange = core.vsrd.SCSelect(clip, sbegin, send, cleansed)
alt = core.rgvs.Repair(scenechange, clip, mode=[16,16,1])
restore = core.rgvs.Repair(cleansed, clip, mode=[16,16,1])
corrected = core.vsrd.RestoreMotionBlocks(cleansed, restore, neighbour=clip, alternative=alt, gmthreshold=70, dist=1, dmode=2, noise=10, noisy=12, grey=0)
clip = core.rgvs.RemoveGrain(corrected, mode=[17,17,1])
clip.set_output()

Lastly, in my humble opinion I think RemoveDirt's namespace "vsrd" should be changed to "rdvs", making it familiar with RemoveGrain's "rgvs" namespace.

---

I wonder why Kassandro chose clense instead of cleanse?

Yes, the calls should have been changed. My bad missing that. I'll look to see why it doesn't load. Last time I tested it it did.

[Mystery Keeper]

Originally Posted by Myrsloik

506k frames later and it still works. I don't think this method of testing will yield anything. I'll add logging of fatal errors and then let you try that build.

Sounds good. Thank you.
Update: got it crash faster while testing with vspipe. I think it said "fwrite error". Testing again with redirecting stderr to file.

[Mystery Keeper]

Ok, it has crashed with "fwrite() call failed" error while ran in windows console with output to stdout.
vspipe D:\Programming\TempLinearApproximate-VapourSynth\build\release-x32\test.vpy - -progress 2>E:\vsout.txt
Here's the log.
It has not crashed for 1500 frames with stdout redirected to file (which wasn't intentional) before I've stopped it.
Used this script to give it harder time.

avisynth_plugins_path = 'E:\\avisynth-plugins\\'
vapoursyth_plugins_path = 'E:\\vapoursynth-plugins\\x32\\'

import vapoursynth as vs
import sys
sys.path.append(vapoursyth_plugins_path + 'E:\\vapoursynth-plugins\\py\\')
core = vs.get_core(threads=8)

core.set_max_cache_size(2500)

import os
for filename in os.listdir(vapoursyth_plugins_path):
	if filename[-4:] != '.dll':
		continue
	core.std.LoadPlugin(vapoursyth_plugins_path + filename)

core.avs.LoadPlugin(path = avisynth_plugins_path + 'mvtools2.dll')

core.std.LoadPlugin(path = 'D:\\Programming\\TempLinearApproximate-VapourSynth\\build\\release-x32\\templinearapproximate.dll')

d2vfile = 'F:\\Video to Process\\Riaru Onigokko 2\\VTS_01_1.d2v'

ret = core.d2v.Source(input=d2vfile)

def pcToTv(input):
	c = core.fmtc.resample (clip=input, css="444")
	c = core.fmtc.matrix (clip=c, mats="601", matd="709")
	c = core.fmtc.resample (clip=c, css="420")
	c = core.fmtc.bitdepth (clip=c, bits=8)
	return c

z = pcToTv(ret)

sys.path.append('D:\\Programming\\TempLinearApproximate-VapourSynth\\')
import MCDenoise

tlamc = MCDenoise.MCDenoise()
tlaArguments = dict(radius=5, BlockSize=8, Overlap=4, SubPel=4, SubPelInterp=2, Search=5, SearchParam=2, PelSearch=4, DCT=10, ThSAD=200)
ret = tlamc.TempLinearApproximate(ret, **tlaArguments)
ret = tlamc.TempLinearApproximate(ret, **tlaArguments)
ret = tlamc.TempLinearApproximate(ret, **tlaArguments)
ret = core.f3kdb.F3kdb(ret, dither_algo=2, grainy=0, grainc=0, keep_tv_range=True)

ret = pcToTv(ret)

ret.set_output()

def absdiff(x, y):
	return min(max(0, 127 + x - y), 255)

unfiltered = core.text.Text(z, "unfiltered")
filtered = core.text.Text(ret, "filtered")
stack = core.std.StackHorizontal([unfiltered, filtered])
diffhist = core.std.Lut2([z, ret], function = absdiff)
#diffhist = core.generic.Levels(diffhist, planes=0, gamma = 3)
diffhist = core.text.Text(diffhist, "Difference")
filteredhist = core.generic.Levels(ret, planes=0, gamma = 2)
filteredhist = core.text.Text(filteredhist, "Filtered amplified")
diffstack = core.std.StackHorizontal([diffhist, filteredhist])
compare = core.std.StackVertical([stack, diffstack])
#compare.set_output()

[Mystery Keeper]

Actually, it might have been an unprintable character issue. Trying again with redirecting stdout to nul.