Vapoursynth

[Myrsloik]

I have too much other stuff to do and I think some of you could help me on the Python side.

Task 1:
If you do something like clip1 + clip2 + clip3 + clip4... this expands to Splice([Splice([Splice([clip1, clip2]), clip3]), clip4])...
It works but becomes a bit inefficient when hundreds of clips are involved and combining it into a single Splice[clip1, clip2, clip3, ...] would be better. Something like a delayed evaluation object for splice operations could be nice to have. Something similar for trim could also be useful. I guess.

The relevant code to modify can be found here. Needs cython to compile but works just like normal python.

Task 2:
I want to flatten things so you don't need to keep a core object around anymore like this:

import vapoursynth as vs
vs.std.BlankClip()

So basically also check the object returned from get_core() for all the attributes not found directly in the vapoursynth module. Cython unfortunately doesn't seem to like the usual Python tricks so I'm a bit stuck there too. Too bad __getattr__ doesn't exist on the module level.

[Sm3n]

Hi,

What would be the best, fastest and effective way to do a SelectRangeEvery like Avisynth does?
I read this https://forum.doom9.org/showthread.p...lectrangeevery but the only thing I can do is a simple SelectEvery.

Should I trim some part of the clip then merge them together or is there an easiest way with the function above?

[Myrsloik]

Originally Posted by Sm3n

Hi,

What would be the best, fastest and effective way to do a SelectRangeEvery like Avisynth does?
I read this https://forum.doom9.org/showthread.p...lectrangeevery but the only thing I can do is a simple SelectEvery.

Should I trim some part of the clip then merge them together or is there an easiest way with the function above?

Why is my previously proposed solution not simple enough?

clip = core.std.SelectEvery(clip = clip, cycle = x, offsets = range(14))

This is functionally identical to SelectRangeEvery. If you want to change the initial offset just trim the input clip. Done!

[Sm3n]

My bad. The values for x and range was way to low. ^^'
It's definitely easy.

Is it possible to tweak QTGMC like in avisynth using mode to get the max speed or vapoursynth does it by itself automaticaly with vs.get_core?

The x264 process is slower with a vpy script than an avs script. It's a simple script:

import vapoursynth as vs
import havsfunc as haf
core = vs.get_core(threads=8)
core.avs.LoadPlugin(r'C:\Program Files (x86)\AviSynth+\plugins64\DGDecodeNV.dll')
video = core.avs.DGSource(r'H:\Videos1.dgi')+core.avs.DGSource(r'H:\Videos2.dgi')+core.avs.DGSource(r'H:\Videos3.dgi')+core.avs.DGSource(r'H:\Videos4.dgi')
video = haf.QTGMC(video, Preset='Medium', TFF=True, FPSDivisor=2).fmtc.resample(kernel="spline36", w=1280, h=720).fmtc.bitdepth(bits=8)
video.set_output()

I think I read that Spline36 resize method is much slower than a simple one. I'm also using it with avisynth. Is it slower than avs's and that would explain the difference of speed?

[Myrsloik]

Originally Posted by Sm3n

My bad. The values for x and range was way to low. ^^'
It's definitely easy.

Is it possible to tweak QTGMC like in avisynth using mode to get the max speed or vapoursynth does it by itself automaticaly with vs.get_core?

The x264 process is slower with a vpy script than an avs script. It's a simple script:

import vapoursynth as vs
import havsfunc as haf
core = vs.get_core(threads=8)
core.avs.LoadPlugin(r'C:\Program Files (x86)\AviSynth+\plugins64\DGDecodeNV.dll')
video = core.avs.DGSource(r'H:\Videos1.dgi')+core.avs.DGSource(r'H:\Videos2.dgi')+core.avs.DGSource(r'H:\Videos3.dgi')+core.avs.DGSource(r'H:\Videos4.dgi')
video = haf.QTGMC(video, Preset='Medium', TFF=True, FPSDivisor=2).fmtc.resample(kernel="spline36", w=1280, h=720).fmtc.bitdepth(bits=8)
video.set_output()

I think I read that Spline36 resize method is much slower than a simple one. I'm also using it with avisynth. Is it slower than avs's and that would explain the difference of speed?

What mode? I don't know what you mean.

Bonus points for not saying how much slower it is so I can speculate why. Different code, different speed. I don't have much to add there.

Try another resizer and find out? Looks easy enough to do...

[Sm3n]

Avisynth can use MTMode so we can make QTGMC process much faster. Sure you already knew. ^^
Can we do that in vapoursynth? For exemple in avisynth QTGMC has this option: EdiThreads=3. Vapoursynth hasn't.

My speed tests:
Avs (QTGMC=medium, spline36) + x264 (veryslow) = 5min to achieve 5.30fps
Vpy (QTGMC=medium, spline36) + x264 (veryslow) = 5min to achieve 4fps

Very significant. I assume CPU doesn't assing the same % to the pipe process.
Sure I can try all I know but I rather ask for all I don't. Maybe there is some options I don't know yet or what.

[Myrsloik]

Originally Posted by Sm3n

Avisynth can use MTMode so we can make QTGMC process much faster. Sure you already knew. ^^
Can we do that in vapoursynth? For exemple in avisynth QTGMC has this option: EdiThreads=3. Vapoursynth hasn't.

My speed tests:
Avs (QTGMC=medium, spline36) + x264 (veryslow) = 5min to achieve 5.30fps
Vpy (QTGMC=medium, spline36) + x264 (veryslow) = 5min to achieve 4fps

Very significant. I assume CPU doesn't assing the same % to the pipe process.
Sure I can try all I know but I rather ask for all I don't. Maybe there is some options I don't know yet or what.

The mtmode stuff only exists in avisynth because there threading is one big hack. Or at least used to be.

What's the cpu usage like? is it 100% for both or less for VS?

[Sm3n]

Originally Posted by Myrsloik

The mtmode stuff only exists in avisynth because there threading is one big hack. Or at least used to be.

What's the cpu usage like? is it 100% for both or less for VS?

Yup, 100% both.

I ran a new test. Vpy = 4.90fps
Now sure for some, difference could be nothing but during an encode taking hours, at the end it might have time wasted.

Thx for your help.

[Myrsloik]

Originally Posted by Sm3n

Yup, 100% both.

I ran a new test. Vpy = 4.90fps
Now sure for some, difference could be nothing but during an encode taking hours, at the end it might have time wasted.

If the difference is that big between runs you really need a better way to measure things. Seriously. With that kind of variation you can't even say that one is faster than the other.

[Sm3n]

Originally Posted by Myrsloik

If the difference is that big between runs you really need a better way to measure things. Seriously. With that kind of variation you can't even say that one is faster than the other.

I'm doing lot of comparison. Have you a good method to measure things to me? Using Avsmeter, encoding 2h show at least once a week et cetera. My knowledge is limited but for that purpose I think I know how my system runs and I'm able to say this is more fastest than that or not. I ran lot of encode I think I understand what is the limit of the speed on case by case.
In fact, next project I have to do I'll be using a vpy script. Usualy an encode takes a bit less than 8h. It will be easy to see the difference I'm sure.

If my cpu can handle X fps in x min using avs vs Y fps in x min using vpy where Y is slowest, I'm pretty sure that avs+x264 process will finish sooner than vpy+x264. Maybe I'm wrong but it makes sense to me.

[TheFluff]

https://zedshaw.com/archive/programm...kill-them-all/

see especially the part about confounding

[Myrsloik]

R34 RC1 YO! Check it out! Basically it fixes the horrible buggy untested mess that R33 was. Should be a safe to use. Just note that the abomination that is temporalsoften no longer is an included plugin.

Installer
Portable 64bit
Portable 32bit

Changes:

r34:
it's now possible to skip installing the visual studio runtimes in the installer to reduce online dependency of installer (installation may not work until manually installed, obviously)
temporalsoften plugin is no longer included in the windows installer, it has been replaced by the misc plugin containing various functions useful for avisynth compatibility
the default max memory usage is now 4GB on 64bit systems
fixed several unpacking issues in avisource introduced in the previous version
added warning if there are any nodes or frames still in use when the core is free
improved avfs prefetch logic for vpy
added premultiplied blending mode to maskedmerge
added premultiply filter to premultiply a clip with alpha
makediff and adddiff no longer pointlessly clamp float to a limited range since there's no good reason to do so
fixed memory leak when the core is freed introduced in the previous version
fixed memory leak in avfs introduced in the previous version
removed rshift and replaced it with a scale argument in sobel and prewitt
fixed sobel and prewitt wrongly passing through the input pixel values
added support for hdr formats and dci-p3 in zimg
optimized planestats
added float support to pemverifier
fixes active region offset and height for interlaced resizing
added support for more different subsampled input formats for vfm (jackoneill)

[~ VEGETA ~]

I am using R33-1 and it is working fine. I use the x64 portable version with python352 and the editor all in one folder as recommended. Now, for updating to newer versions, should I just copy and replace VS files (excluding plugins folder)?

[Myrsloik]

Originally Posted by ~ VEGETA ~

I am using R33-1 and it is working fine. I use the x64 portable version with python352 and the editor all in one folder as recommended. Now, for updating to newer versions, should I just copy and replace VS files (excluding plugins folder)?

Including the plugins dir. Overwriting an existing version is fine.

[dipje]

The HDR stuff seems to work nicely. Able to convert some clips to something that is quite OK to look at on regular screens. I do a little gamma tweak (like 1.2 or 1.3) to lift the shadows a bit at the end with a nominal luminance somewhere between 400 and 600 and so far so good.

That being said, can someone help me to get something like highlight compression or shadow boosting without altering the entire gamma?
A bit like Photo-correcting software does.

Is something possible? with Expr or something? Or make / define a curve myself with float values that map values close-to 1.0 and over 1.0 to a sane value within the 0.0 - 1.0 range?

And / or, is it possible to load and apply 3d Cube lut files?

[Myrsloik]

Originally Posted by dipje

The HDR stuff seems to work nicely. Able to convert some clips to something that is quite OK to look at on regular screens. I do a little gamma tweak (like 1.2 or 1.3) to lift the shadows a bit at the end with a nominal luminance somewhere between 400 and 600 and so far so good.

That being said, can someone help me to get something like highlight compression or shadow boosting without altering the entire gamma?
A bit like Photo-correcting software does.

Is something possible? with Expr or something? Or make / define a curve myself with float values that map values close-to 1.0 and over 1.0 to a sane value within the 0.0 - 1.0 range?

And / or, is it possible to load and apply 3d Cube lut files?

1. Loading cube luts is eternally on my todo list but don't wait for it

2. If you only want 1d adjustment you can definitely do that with expr if you don't mind creating your own expression

3. I have no idea what algorithm photo correction software uses (and I can't google my way to it since you didn't list software+mode) but I suspect it's closer to local constrast enhancement than a 1d lut adjustment

[~ VEGETA ~]

Originally Posted by Myrsloik

Including the plugins dir. Overwriting an existing version is fine.

Well, I have done some effort collecting plugins into that folder, and I have another folder called "scripts" beside it.

The plugins folder comes empty anyway. If you mean the core plugins then fine, I can replace it.

[Myrsloik]

Originally Posted by ~ VEGETA ~

Well, I have done some effort collecting plugins into that folder, and I have another folder called "scripts" beside it.

The plugins folder comes empty anyway. If you mean the core plugins then fine, I can replace it.

Yes, that's what I meant.

[VS_Fan]

Originally Posted by dipje

That being said, can someone help me to get something like highlight compression or shadow boosting without altering the entire gamma?
A bit like Photo-correcting software does.

Some time ago I got amazing results in AVIsynth with the "HDR AGC plugin".

Also, have you tried "Retinex" plugin for VapourSynth?

[Myrsloik]

Changes from RC1:
Pismo PFM runtime included in the installer to make AVFS usage easier
AVFS supports avs+ high bitdepth output
Super minor fixes nobody will ever notice

R34 RC2
Portable 64bit
Portable 32bit