Vapoursynth

[feisty2]

I see, so I should copy the string with an owning std::string.

[Myrsloik]

Originally Posted by feisty2

I see, so I should copy the string with an owning std::string.

Yes, always.

[Boulder]

If I have a VFR video clip, say like part A deinterlaced to 25fps and part B bobbed to 50fps in the script and the two parts combined with A+B - what is the proper way to get the correct timecodes out with vspipe? Should I use AssumeFPS to set the framerate to 25fps (which I will use when I feed it to x265) or leave it as it is?

[Myrsloik]

R49 released!

[Myrsloik]

Originally Posted by Boulder

If I have a VFR video clip, say like part A deinterlaced to 25fps and part B bobbed to 50fps in the script and the two parts combined with A+B - what is the proper way to get the correct timecodes out with vspipe? Should I use AssumeFPS to set the framerate to 25fps (which I will use when I feed it to x265) or leave it as it is?

It's been a long time since I did this so it could be wrong but generally you simply splice the two different fps clips (mismatch=1 in splice) and then use it as input to x265. Make sure to use the vspipe timecode option and after the encode is done mux in the proper timecodes.

Or that's how I think it should work.

[Boulder]

Originally Posted by Myrsloik

It's been a long time since I did this so it could be wrong but generally you simply splice the two different fps clips (mismatch=1 in splice) and then use it as input to x265. Make sure to use the vspipe timecode option and after the encode is done mux in the proper timecodes.

Or that's how I think it should work.

Doesn't seem to be working properly I fed the VFR clip into x265 and told it to expect 25fps input (otherwise it would simply crash). The clip was produced by splicing with the + operator in Vapoursynth.

The timecodes have two steps, 20ms and 40ms in the 50fps parts. The 25fps parts have 40ms and 80ms steps.

# timecode format v2
0.000000
20.000000
60.000000
80.000000
120.000000
.
.
.
46600.000000
46680.000000
46720.000000
46800.000000
46840.000000
46920.000000
46960.000000

I have some similar old Matroska files in which I have encoded the video track at 50fps (at least according to MediaInfo) and created the timestamp v1 file manually. I now extracted the timestamps and they look like this, first the 50fps part and the latter one is from a 25fps part:

# timestamp format v2
0
20
40
60
.
.
.
408000
408040
408080
408120
408160

[DJATOM]

Boulder
You probably doing it wrong.
Yesterday I had hybrid video file encoded as MBAFF (no actual combed frames, but it's 30000/1001i), and some sections are upscaled from 24000/1001p. So I need to decimate only certain sections and produce VFR video.
I'll describe my solution bellow and how to actually get valid timecodes (Myrs answer seems not so clear for you). The whole process is described to understand my workflow and most likely irrelevant for you, but you can adapt that to your case and use only what you need from it.

First, I have to check with VDecimate(dryrun=True) if scene actually telecined. The result for my video was

scenes = [0, 80, 173, 269, 366, 462, 561, 609, 657, 705, 1255, 1306]

Video starts from 24p scene, so I have to decimate every odd section.

vfr_pool = list()
for pos, start in enumerate(scenes, 1): # start from 1 for simplicity of the algo
	if pos >= len(scenes): # last frame in list is the end of last section, so I want to terminate loop on it
		break
	end = scenes[pos] 
	if pos&1: # decimate only 1, 3, 5, 7, etc. sections
		vfr_pool.append(core.vivtc.VDecimate(source[start:end]))
	else:
		vfr_pool.append(source[start:end])

And finally splice those scenes into one VFR video

vfr_clip = core.std.Splice(vfr_pool)
vfr_clip.set_output()

Now it's ready to encode,

vspipe -t mytimecodes.txt -y script.vpy - | x265 ... --y4m --fps 24000/1001 -

and after encoding use mytimecodes.txt to mux into mkv video. Result should be VFR and properly sync with audio.
I really want tdecimate's mode 4/5 solution in vapoursynth, but it seems like no one want to port that.

[Boulder]

Thank you, I will take a look at your method

I used this kind of approach for creating the test clip, the first and third parts are bobbed.

clp = core.dgdecodenv.DGSource(r"F:\Temp\Captures\monty\monty_s01e01.dgi", cl=244, cr=244)

result = haf.QTGMC(core.std.Trim(clp, 0,719), Preset='very fast', Search=5, SearchParam=8, PelSearch=8, ChromaMotion=True, ChromaNoise=False, SourceMatch=2, Lossless=2, EZKeepGrain=0.4, Sharpness=0.1, TR2=0, TFF=True)
result = result + haf.QTGMC(core.std.Trim(clp, 720,2890), Preset='very fast', Search=5, SearchParam=8, PelSearch=8, ChromaMotion=True, ChromaNoise=False, SourceMatch=2, Lossless=2, EZKeepGrain=0.4, Sharpness=0.1, TR2=0, TFF=True, InputType=2)
result = result + haf.QTGMC(core.std.Trim(clp, 2891,3935), Preset='very fast', Search=5, SearchParam=8, PelSearch=8, ChromaMotion=True, ChromaNoise=False, SourceMatch=2, Lossless=2, EZKeepGrain=0.4, Sharpness=0.1, TR2=0, TFF=True)
result = result + haf.QTGMC(core.std.Trim(clp, 3936,4428), Preset='very fast', Search=5, SearchParam=8, PelSearch=8, ChromaMotion=True, ChromaNoise=False, SourceMatch=2, Lossless=2, EZKeepGrain=0.4, Sharpness=0.1, TR2=0, TFF=True, InputType=2)

result.set_output()

Thinking a bit further, I think I could use some Python scripting to autofill the Trims as I've collected all the parts like this.

#0, 719 i
#720, 2890 p
#2891, 3935 i
#3936, 4428 p
#4429, 6676 i
#6677, 6834 p
#6835, 10567 i
#10568, 12260 p
#12261, 23093 i
#23094, 23991 p
#23992, 24154 i
#24155, 25876 p
#25877, 26664 i
#26665, 30805 p
#30806, 31140 i
#31141, 40655 p
#40656, 43509 i
#43510, 43599 p
#43600, 44268 i
#44269, 44403 p
#44404, 44831 i
#44832, 45795 p
#45796, 47686 i

[DJATOM]

Apparently timecodes produced with vspipe are weird. Working solution is

import easyvfr # https://gist.github.com/chikuzen/5005590 (usage: http://csbarn.blogspot.com/2013/02/e...poursynth.html)
<...>
clp = core.dgdecodenv.DGSource(r"F:\Temp\Captures\monty\monty_s01e01.dgi", cl=244, cr=244)
qtgmc_common_opts = {'Preset': 'very fast', 'Search': 5, 'SearchParam': 8, 'PelSearch': 8, 'ChromaMotion': True, 'ChromaNoise': False, 'SourceMatch': 2, 'Lossless': 2, 'EZKeepGrain': 0.4, 'Sharpness': 0.1, 'TR2': 0, 'TFF': True}
clips = []
clips.append(haf.QTGMC(core.std.Trim(clp, 0,719), **qtgmc_common_opts))
clips.append(haf.QTGMC(core.std.Trim(clp, 720,2890), **qtgmc_common_opts, InputType=2))
clips.append(haf.QTGMC(core.std.Trim(clp, 2891,3935), **qtgmc_common_opts))
clips.append(haf.QTGMC(core.std.Trim(clp, 3936,4428), **qtgmc_common_opts, InputType=2))
vfr = easyvfr.EasyVFR(clips, base_num=25, base_den=1)
vfr.write_timecode(r'test.tc.txt')
vfr.splice_clips().set_output()

That way we have valid timecodes, after converting to v1 (tcConv) the result is

# timecode format v1
Assume 25.000000
0,1439,50.000000
3612,5701,50.000000

[Boulder]

Thanks a lot, that probably saves me a huge amount of manual work

[feisty2]

I updated to r49 and weird things happened.
I ran the same std.Convolution script for speed test and the script ran for 497.17fps, but it was 2400+fps before the update!
my custom GaussBlur filter also ran a lot slower, from 1800+fps before update to now 483.92fps (gcc -O3)

[Are_]

std.Convolution with same parameters as you runs at 2384 fps here (Linux), maybe Windows build is compiled without optimizations?
I also did try to run it with core.std.SetMaxCPU('none') but it made no difference? Maybe I'm too sleepy and I'm doing something wrong.

import vapoursynth as vs
core = vs.get_core()
core.std.SetMaxCPU('none')
clip = core.std.BlankClip(format=vs.GRAYS, length=100000, fpsnum=24000, fpsden=1001, keep=True)
clip = core.std.Convolution(clip, matrix=[1,2,1,2,4,2,1,2,1])
clip.set_output()

[feisty2]

Originally Posted by Are_

std.Convolution with same parameters as you runs at 2384 fps here (Linux), maybe Windows build is compiled without optimizations?
I also did try to run it with core.std.SetMaxCPU('none') but it made no difference? Maybe I'm too sleepy and I'm doing something wrong.

import vapoursynth as vs
core = vs.get_core()
core.std.SetMaxCPU('none')
clip = core.std.BlankClip(format=vs.GRAYS, length=100000, fpsnum=24000, fpsden=1001, keep=True)
clip = core.std.Convolution(clip, matrix=[1,2,1,2,4,2,1,2,1])
clip.set_output()

could you compile this with GCC10 -Ofast and run a speed test?

[Are_]

I will give it a try in about one hour or so, I don't have GCC-10 installed and it will take a while here on Gentoo.

[ChaosKing]

Originally Posted by Are_

std.Convolution with same parameters as you runs at 2384 fps here (Linux), maybe Windows build is compiled without optimizations?
I also did try to run it with core.std.SetMaxCPU('none') but it made no difference? Maybe I'm too sleepy and I'm doing something wrong.

import vapoursynth as vs
core = vs.get_core()
core.std.SetMaxCPU('none')
clip = core.std.BlankClip(format=vs.GRAYS, length=100000, fpsnum=24000, fpsden=1001, keep=True)
clip = core.std.Convolution(clip, matrix=[1,2,1,2,4,2,1,2,1])
clip.set_output()

R49 quick test on ryzen 2600, seems ok
vspipe.exe .\bla.vpy .
Output 100000 frames in 6.20 seconds (16138.35 fps) # without core.std.SetMaxCPU('none')
Output 100000 frames in 12.73 seconds (7856.92 fps) #with core.std.SetMaxCPU('none')

[sl1pkn07]

with/without dual xeon (:/) (linux, -02)

└───╼  vspipe test.vpy /dev/null
Output 100000 frames in 15.89 seconds (6293.29 fps)
└───╼  vspipe test.vpy /dev/null
Output 100000 frames in 9.40 seconds (10642.37 fps)

seems Konsole (KDE terminal) do strange things

lanuched the same in yakuake (based on konsole, not use as bakend)

┌─┤[$]|[sl1pkn07]|[sL1pKn07]|[~/aplicaciones/vapoursynth-git]|
└───╼  cat test.vpy 
import vapoursynth as vs
core = vs.get_core()
core.std.SetMaxCPU('none')
clip = core.std.BlankClip(format=vs.GRAYS, length=100000, fpsnum=24000, fpsden=1001, keep=True)
clip = core.std.Convolution(clip, matrix=[1,2,1,2,4,2,1,2,1])
clip.set_output() 
┌─┤[$]|[sl1pkn07]|[sL1pKn07]|[~/aplicaciones/vapoursynth-git]|
└───╼  vspipe test.vpy /dev/null
Output 100000 frames in 8.39 seconds (11923.86 fps)
┌─┤[$]|[sl1pkn07]|[sL1pKn07]|[~/aplicaciones/vapoursynth-git]|
└───╼  cat test.vpy 
import vapoursynth as vs
core = vs.get_core()
#core.std.SetMaxCPU('none')
clip = core.std.BlankClip(format=vs.GRAYS, length=100000, fpsnum=24000, fpsden=1001, keep=True)
clip = core.std.Convolution(clip, matrix=[1,2,1,2,4,2,1,2,1])
clip.set_output() 
┌─┤[$]|[sl1pkn07]|[sL1pKn07]|[~/aplicaciones/vapoursynth-git]|
└───╼  vspipe test.vpy /dev/null
Output 100000 frames in 7.32 seconds (13656.67 fps)
┌─┤[$]|[sl1pkn07]|[sL1pKn07]|[~/aplicaciones/vapoursynth-git]|
└───╼

htop with vspipe in konsole shot
htop with vspipe in yakuake shot

[leon]

Why does R49 installer force the download of vc++ 2019?

[Myrsloik]

Originally Posted by leon

Why does R49 installer force the download of vc++ 2019?

Because your runtimes are old and you didn't unchecked the option.

[leon]

You mean it checks for the installed runtimes? Because I'd already installed both x86 and x64 versions of it. You're right there's an option for it which I somehow missed.
Thank you.

BTW, I installed Python 3.8 in Program Files and moved its path to the end of PATH variable so that I'd be able to use the already installed 3.7.4 version, things appear to work so far, but I thought I'd ask here whether that's OK or not.

P.S. I updated from R45 so I thought I'd see some speed improvements considering AVX2 has been added since that version, but I got almost the same speeds, so was wondering if there's a way to check whether I was doing something wrong and that AVX2 was actually being used.

[Myrsloik]

Originally Posted by leon

You mean it checks for the installed runtimes? Because I'd already installed both x86 and x64 versions of it. You're right there's an option for it which I somehow missed.
Thank you.

BTW, I installed Python 3.8 in Program Files and moved its path to the end of PATH variable so that I'd be able to use the already installed 3.7.4 version, things appear to work so far, but I thought I'd ask here whether that's OK or not.

P.S. I updated from R45 so I thought I'd see some speed improvements considering AVX2 has been added since that version, but I got almost the same speeds, so was wondering if there's a way to check whether I was doing something wrong and that AVX2 was actually being used.

Yes, it checks for the exact version of installed runtimes. Note that the 2019 ones get updated every few months and I install the most recent minor update too. That's why it appears to you like I'm "pointlessly" installing it.

Don't do that python mess. You've most likely ended up using the R48 python module and things just work by accident.