a dummy's query, is fps drop expected from core.std.SelectEvery ?

[hydra3333]

Hello. Dummy seeking advice. In the script below, this line
video = core.std.SelectEvery(video, cycle=4, offsets=[0, 3])
in the "re-interlacing" part of script, drops fps by half, from 20fps down to 10fps, when piping to x264 and encoding as interlaced.

Taking longer elapsed, sure as there's more material to encode, but dropping the fps by half ?
I must be thinking about it wrongly, which seems likely, unless there's something very slow about that line.
Is someone able to clarify ?
import vapoursynth as vs
# snipped imports and loadplugins
def main():
core = vs.get_core(accept_lowercase=True)
video = core.d2v.Source(r'T:\HDTV\WDTVlive\MP4-VS\x64-1\0-test.d2v')
video = core.std.Trim(video,first=0,length=500)
# INTERLACED-AWARE DEBLOCKER IN HERE
video = core.std.SeparateFields(video, tff=True)
video = haf.Deblock_QED(video)
video = haf.Weave(video, tff=True)
video = mvs.AssumeTFF(video)
# QTGMC IN HERE (SLMODE=1 is spatial, SLMODE=2 is temporal)
video = haf.QTGMC(video, TFF=True, Preset="Very Slow", Sharpness=1.2, SLMode=2, EZKeepGrain=1.2, NoiseProcess=2) # result is double framerate progressive, so re-interlate it later
# DEHALO VIDEO
video = core.fmtc.bitdepth(video,bits=32,fulls=False,fulld=True) #convert to 32bit float. Specify if the clip is full-range (True) or TV-range (False). fulls is for input, fulld for output.
#video = Vine.Dehalo(video, radius=[2, None])
video = core.fmtc.bitdepth(video,bits=8,fulls=True,fulld=False) #convert to 8bit Integer. Specify if the clip is full-range (True) or TV-range (False). fulls is for input, fulld for output.
# RE-INTERLACE VIDEO from double-framerate back down to normal. this is right for 576p50 to 576i25
video = core.std.SeparateFields(video, tff=True)

# if comment out the next line we get circa 20fps if we uncomment it then we drop way down to only circa 10fps ... should we expect that ?
video = core.std.SelectEvery(video, cycle=4, offsets=[0, 3]) # looks like avisynth

video = haf.Weave(video, tff=True)
video = mvs.AssumeTFF(video)
video.set_output()
return True
# ... functions here snipped
main()


x64 portable.

[Myrsloik]

That's the documented behavior, yes.

[jackoneill]

Are you talking about encoding speed or the frame rate of the finished video? It's not entirely clear.

[hydra3333]

Apology, the speed with which x264 encodes the resulting clips, in fps. x264 reports the speed at which it is performing its encoding of the video as it goes along, in fps, unless i've bolluxed it up and it means something else.

I'm assuming for purpose of testing that the x264 speed of encoding is rather constant and should be approximately the same for both tests ... and that the halving of the fps (the rate at which the x264 encoding is performed) is due to that script-line taking a long time to process within vapoursynth, a "quite significant choke point in the script" if you will.

[jackoneill]

The previous filter is SeparateFields, which turns every input frame into two frames, at practically zero cost. Also, the output frames have half the height of the input frames. I imagine x264 could very well encode this in the same time it takes to encode the original video.

SelectEvery reduces the number of frames back to the original number, thus reducing the apparent speed as well.

At least this is what I think happens. Try encoding a handful of frames with and without that SelectEvery. Put a Trim right after the source filter. Pick a reasonably low number of frames, something that won't make you wait hours for the results. I don't know if vspipe's --progress report is reliable in this case, so just look at the clock or something.

[hydra3333]

OK, seeking your view - does "core.std.SelectEvery(video, cycle=4, offsets=[0, 3])" seem expensive in fps output-rate terms ? Just wondering.

Simple without and with:
3212.22 fps vs 1611.67 fps
and
50.64 fps vs 41.35 fps
A different script which qtgmc'd and re-interlaced about halved the fps from 20fps down to 10fps, when using "core.std.SelectEvery(video, cycle=4, offsets=[0, 3])".

core = vs.get_core(accept_lowercase=True)
def main():
video = core.d2v.Source(r'T:\HDTV\WDTVlive\MP4-VS\x64-1\0-test.d2v')
#video = core.std.SelectEvery(video, cycle=4, offsets=[0, 3]) # looks like avisynth
video.set_output()
return True
main()

"C:\SOFTWARE\Vapoursynth-x64\VSPipe.exe" --progress "T:\HDTV\WDTVlive\MP4-VS\x64-1\0-test.mpg.2016.08.27.10.32.36.05.vpy" .
# --- Output 45810 frames in 14.26 seconds (3212.22 fps)

"C:\SOFTWARE\Vapoursynth-x64\VSPipe.exe" --progress "T:\HDTV\WDTVlive\MP4-VS\x64-1\0-test.mpg.2016.08.27.10.40.47.62.vpy" - --y4m 2>".\vspipe-progress.txt" | "C:\SOFTWARE\ffmpeg\0-homebuilt-x64\x264-mp4.exe" - --stdin y4m --thread-input --frames 45810 --profile high --level 4.1 --preset slow --interlaced --tff --no-cabac --crf 16 --sar 64:45 --colormatrix bt470bg -o "s:\HDTV\WDTVlive\MP4-VS\x64-1\0-test-temp.2016.08.27.10.40.47.62.h264"
# --- Output 1000 frames in 18.07 seconds (55.36 fps)
# --- x264 encoded 1000 frames, 50.64 fps, 5539.51 kb/s


then
core = vs.get_core(accept_lowercase=True)
def main():
video = core.d2v.Source(r'T:\HDTV\WDTVlive\MP4-VS\x64-1\0-test.d2v')
video = core.std.SelectEvery(video, cycle=4, offsets=[0, 3]) # looks like avisynth
video.set_output()
return True
main()

"C:\SOFTWARE\Vapoursynth-x64\VSPipe.exe" --progress "T:\HDTV\WDTVlive\MP4-VS\x64-1\0-test.mpg.2016.08.27.10.32.36.05.vpy" .
# --- Output 22905 frames in 14.21 seconds (1611.67 fps)

"C:\SOFTWARE\Vapoursynth-x64\VSPipe.exe" --progress "T:\HDTV\WDTVlive\MP4-VS\x64-1\0-test.mpg.2016.08.27.10.45.16.08.vpy" - --y4m 2>".\vspipe-progress.txt" | "C:\SOFTWARE\ffmpeg\0-homebuilt-x64\x264-mp4.exe" - --stdin y4m --thread-input --frames 45810 --profile high --level 4.1 --preset slow --interlaced --tff --no-cabac --crf 16 --sar 64:45 --colormatrix bt470bg -o "s:\HDTV\WDTVlive\MP4-VS\x64-1\0-test-temp.2016.08.27.10.45.16.08.h264"
# --- Output 500 frames in 10.52 seconds (47.53 fps)
# --- x264 encoded 500 frames, 41.35 fps, 4575.13 kb/s


x64 portable, i3820 16Gb memory, input PAL 576i

[AzraelNewtype]

These test results are saying it takes about 14.2 seconds to decode the mpeg2 stream pointed to by that d2v, and SelectEvery isn't affecting the speed at all. If you discard the half the frames, and only calculate the frames/second at the very end, you will obviously be dividing a number half as big by the same amount of time. You are still processing all 45810 frames of input at the same rate. This is a non-issue.