Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 6th October 2021, 22:42   #1  |  Link
markfilipak
Registered User
 
markfilipak's Avatar
 
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 263
MV interpolation of mixed TV+cinema to 120fps progressive

Transcoding a mix of TV and telecined video is the hardest task. It is the last piece in my task list. I present the issues, my nomenclature, some ffmpeg, and some issues.
I would appreciate your thoughts and some clues to overcoming the issues.

Code:
"fps" means "frames per second", "sps" means "scans per second", "pps" means "pictures per second".
29.9fps[59.9sps]        (read: "29.9 fps of 59.9 sps")                    is interlaced scans (e.g. NTSC).
29.9fps[2-3[24pps]]     (read: "29.9 fps of 2-3 pull-down of 24 pps")     is hard telecined (2-3 pull-down) cinema.
29.9fps[3-2[24pps]]     (read: "29.9 fps of 3-2 pull-down of 24 pps")     is hard telecined (3-2 pull-down) cinema.
29.9fps[3-3-2-2[24pps]] (read: "29.9 fps of 3-3-2-2 pull-down of 24 pps") is hard telecined (3-3-2-2 pull-down) cinema.
29.9fps[2-2-3-3[24pps]] (read: "29.9 fps of 2-2-3-3 pull-down of 24 pps") is hard telecined (2-2-3-3 pull-down) cinema.
'c' denotes combed frame, '-' denotes uncombed frame, time flows left to right.

c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c   ...Scans: 29.9fps[59.9sps].
- - c c - - - c c - - - c c - - - c c - - - c c - - - c c - - - c c - -   ...Hard Telecine: 29.9fps[2-3[24pps]].
- - - c - - - - c - - - - c - - - - c - - - - c - - - - c - - - - c - -   ...Hard Telecine: 29.9fps[2-2-3-3[24pps]].
- c c - - - c c - - - c c - - - c c - - - c c - - - c c - - - c c - - -   ...Hard Telecine: 29.9fps[3-2[24pps]].
- c - - - - c - - - - c - - - - c - - - - c - - - - c - - - - c - - - -   ...Hard Telecine: 29.9fps[3-3-2-2[24pps]].
- - - - - c c - - - - - c c - - - - - - - - - - - c c - - - - - - - c c   ...Hard Telecine: Wacky pull-down (seen in a few videos, e.g. "PASSION FISH");
          \____________\___(2 combed frames alternate with 5 / 7.. uncombed frames) -- always odd, always 2 combed frames, seemingly random intervals.
c c c c c c c c c c c c c c c c c c c - - c c - - - c c - - - c c - - -   ...Mixed Scans+Hard Telecine: e.g. 29.9fps[[59.9sps]+2-3[24pps]].
                                       \
                                        A well cut, 'Hard Telecine' segment always begins with an uncombed frame.
                                        A poorly cut, 'Hard Telecine' segment may suffer 1 or 2 combed frames coinciding with scene change.
For 29.9fps[59.9sps] segments:
Code:
  SET   UNWEAVE=settb=expr=1001/30000,setpts=expr=N,separatefields,split[1][2]
  SET    FIELD1=[1]select=eq(mod(n\,2)\,0),minterpolate=fps=120000/1001:mi_mode=mci:scd=fdiff,settb=expr=1/120,setpts=expr=N,split[3][4]
  SET    FIELD2=[2]select=eq(mod(n\,2)\,1),minterpolate=fps=120000/1001:mi_mode=mci:scd=fdiff,settb=expr=1/120,setpts=expr=N[5]
  SET       BOB=[3]select=lt(n\,2)[6]
  SET  AFTERBOB=[5]select=gt(n\,1)[7]
  SET     MERGE=[6][7]tinterlace=mode=merge[8]
  SET NEWFIELD2=%FIELD2%,%BOB%,%AFTERBOB%,%MERGE%
  SET     WEAVE=[4][8]weave[v]
  -filter_complex "%UNWEAVE%,%FIELD1%,%NEWFIELD2%,%WEAVE%"
The 2 fields are separated and independently 4x MV converted to 120fps.
As the 1st field stream passes through unaltered, duplicates of its 1st 2 frames are split off.
The 2nd field stream is delayed by 2 frames and merged with the 2 split off frames from field 1. That process, 1, bobs the 1st 2 frames, and 2, shifts field 2's frames to temporally align them with field 1, thereby making progressive frames out of scan fields with zero cosmetic filtering!
The 2 streams are then woven.

For 29.9fps[2-3[24pps]] segments:
Code:
  -filter_complex "settb=expr=1001/30000,setpts=expr=N,detelecine,minterpolate=fps=120000/1001:mi_mode=mci:scd=fdiff,settb=expr=1/120,setpts=expr=N[v]"
The result is 5x MV interpolation of the detelecined source.

To pull this off, I need a flag, 'combed', so that, after I combine filter graph sections, I know when to switch between them. Cu Selur suggested I try 'tdm.IsCombed' (https://github.com/HomeOfVapourSynth...ynth-TDeintMod) but I think it scans an entire clip and can't flag segments.

And I'd also like to replace 'minterpolate' with pipes to/from VapourSynth's superior MV interpolation: Super(), Analyse(), SmoothFps(), but I don't know whether a pipe in the middle of a filter graph is possible -- I suspect not -- or alternatively, whether ffmpeg can support 2 filter graphs with to/from pipes between them -- I've never tried 2 'filter_complex's in one ffmpeg command).
markfilipak is offline   Reply With Quote
Old 7th October 2021, 16:30   #2  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,421
Cu = See you,... Name is just Selur.
Quote:
I think it scans an entire clip and can't flag segments.
Haven't used 'tdm.IsCombed' for ages, but as far as I know it creates a property clip which containes the combed info on a per frame basis,.. but that probably doesn't really help when using ffmpeg.
Quote:
but I don't know whether a pipe in the middle of a filter graph is possible -- I suspect not -
Nope, at least as I know that there is no filter in ffmpeg which would allow to use a Vapoursynth script for filtering.
-> either ffmpeg or Vapoursynth

Cu Selur
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 8th October 2021, 05:43   #3  |  Link
markfilipak
Registered User
 
markfilipak's Avatar
 
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 263
Quote:
Originally Posted by Selur View Post
Cu = See you,... Name is just Selur.

Haven't used 'tdm.IsCombed' for ages, but as far as I know it creates a property clip which containes the combed info on a per frame basis,..
Howdy! I'm not sure what a 'property clip' is. Is it, 1, a scoreboard that can be used as an auxiliary reference source in a 2nd pass, or 2, a text list similar to what ffprobe or MediaInfo produces, or 3, a 'clip' (meaning: a video)?
markfilipak is offline   Reply With Quote
Old 8th October 2021, 19:38   #4  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 6,421
Since it's passed to FrameEval (http://www.vapoursynth.com/doc/funct...#std.FrameEval)
Code:
std.FrameEval(vnode clip, func eval[, vnode[] prop_src, vnode[] clip_src])
in the example (https://github.com/HomeOfVapourSynth...ge-of-iscombed) it seesm to be an array of video nodes.
My guess is that you best ask one of the authors about further details.
I would have expected it to be just one clip where '_Combed' frame propery (http://www.vapoursynth.com/doc/apire...ame-properties) are set for the frames. (No clue why it's a vnode[].)

Cu Selur
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 8th October 2021, 22:06   #5  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 376
Quote:
Originally Posted by Selur View Post
Since it's passed to FrameEval
...
it seesm to be an array of video nodes.
...
I would have expected it to be just one clip where '_Combed' frame propery (http://www.vapoursynth.com/doc/apire...ame-properties) are set for the frames. (No clue why it's a vnode[].)
FrameEval is a general function for any kind of per frame processing. Sometimes you need to access more clips (and their frame properties) than just one, that's why it's a vnode[]. But you don't *have to* input more than one clip if you don't need to.
zorr is offline   Reply With Quote
Old 8th October 2021, 22:41   #6  |  Link
markfilipak
Registered User
 
markfilipak's Avatar
 
Join Date: Jul 2016
Location: Mansfield, Ohio (formerly San Jose, California)
Posts: 263
Quote:
Originally Posted by Selur View Post
Since it's passed to FrameEval (http://www.vapoursynth.com/doc/funct...#std.FrameEval) ...
Oh, My! Is that what I think it is? Is that a function to edit frames, fields, etc. (all of what I call "mechanical" video properties) in a way that's analogous to what I do currently in an FFmpeg filter graph?

(I prominantly saved those links.)

Now, for an ignorant question (I have no pride) ...
Quote:
Code:
std.FrameEval(vnode clip, func eval[, vnode[] prop_src, vnode[] clip_src])
in the example (https://github.com/HomeOfVapourSynth...ge-of-iscombed) it seesm to be an array of video nodes.
My guess is that you best ask one of the authors about further details.
I would have expected it to be just one clip where '_Combed' frame propery (http://www.vapoursynth.com/doc/apire...ame-properties) are set for the frames. (No clue why it's a vnode[].)
What is a vnode?

PS: I may just wind up retiring what I know/do in FFmpeg in favor/favour of VS. I'll need some help with it. Are you folks game?

PPS: Just to let you know that I'm not lazy, I searched for "What is a vnode" and got a million irrelevant hits, and for "What is a vapoursynth vnode" and got everything vapoursynth but not vnode; I searched the vapoursynth install directory and the vapoursynth downloaded directory for anything/everything "vnode" and got 4 '.dll's. That means that the vapoursynth documentation '.html' does not contain the string "vnode". ...Oh, and I searched for ANSI, UTF-8, and UTF-16, non-case, non-wholewords.

Last edited by markfilipak; 9th October 2021 at 00:59.
markfilipak is offline   Reply With Quote
Old 9th October 2021, 01:29   #7  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 183
Someone finally please help him! He desperately wants to learn avisynth and vapoursynth and forget ffmpeg.
richardpl is offline   Reply With Quote
Old 9th October 2021, 02:28   #8  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 222
vnode is just a name, label in general, it means vapoursynth video clip if refering in a new docs.

vapoursynth clip up until recently was always considered always vapoursynth.VideoNode type
Code:
import vapoursynth as vs
clip = vs.core.lsmas.LibavSMASHSource('video.mp4')
print(type(clip))
if isinstance(clip, vs.VideoNode):
    print('It is python object vapoursynth.VideoNode that is referred as vnode in docs now.')
it prints:
<class 'vapoursynth.VideoNode'>
It is python object vapoursynth.VideoNode that is referred as vnode in docs now.

It just started recently, because audio support was added for vapoursynth, so kind of making sure what docs refer to, video clip or audio clip:
Code:
import vapoursynth as vs
my_video = vs.core.lsmas.LibavSMASHSource('video.mp4')
my_audio = vs.core.bas.Source('video.mp4')
if isinstance(my_video, vs.VideoNode):
    print('It is vapoursynth.VideoNode object that is called vnode in new docs for R55 and above')
if isinstance(my_audio, vs.AudioNode):
    print('It is vapoursynth.AudioNode object that is called anode in new docs for R55 and above')
So yes , it is kind of confusing right now, that link for new docs for API4 , R55 release and above uses vnode reference for video clip and anode for audio clip.

Last edited by _Al_; 9th October 2021 at 02:52.
_Al_ is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.