Remove pulldown from alleged progressive source

[asarian]

What is the best way to remove the pulldown from this video? eac3to tells me "The MPEG2 stream is a mixture of progressive and interlaced encoding." And I don't see any option for pulldown in QTGMC. Little confused here. :)


Video
ID : 1
Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : Variable
Codec ID : V_MPEG2
Codec ID/Info : MPEG 1 or 2 Video
Bit rate mode : Variable
Bit rate : 5 189 Kbps
Maximum bit rate : 9 800 Kbps
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Original frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Scan order : 2:3 Pulldown
Compression mode : Lossy
Time code of first frame : 01:00:00:00
Time code source : Group of pictures header
GOP, Open/Closed : Open
GOP, Open/Closed of first frame : Closed
Default : No
Forced : No

[johnmeyer]

What makes you think the video itself contains pulldown? It may already be progressive, and the "2:3 pulldown" being reported may be nothing more than the pulldown flag. I say this because the frame rate is being reported as 23.976.

If you have any question, do a lossless cut of a few seconds of high motion and upload that so people can take a look at it.

[asarian]

What makes you think the video itself contains pulldown? It may already be progressive, and the "2:3 pulldown" being reported may be nothing more than the pulldown flag. I say this because the frame rate is being reported as 23.976.

If you have any question, do a lossless cut of a few seconds of high motion and upload that so people can take a look at it.


Thanks for your reply.

tsMuxer lists it as 720:480i, 29.97fps. Hence my confusion. :) I uploaded a 2secs sample, though, here:

What pulldown is this? (https://www.dropbox.com/s/0x7q775x150oezc/VTS_01_1.m2ts?dl=0)

[Sharc]

It looks like telecined to me.
I would suggest:
TFM().tdecimate()
QTGMC(InputType=1) #stabilizing & cleaning
which returns 23.976 progressive frames per second

(It would be helpful to get a segment with action/movement rather than a static scene)

[Music Fan]

Yes, that's too short and not enough animated to see exactly what it is.
For Gspot, this is 23.976 fps with a pulldown flag, you can remove it with TsMuxer.

[asarian]

It looks like telecined to me.
I would suggest:
TFM().tdecimate()
QTGMC(InputType=1) #stabilizing & cleaning
which returns 23.976 progressive frames per second

(It would be helpful to get a segment with action/movement rather than a static scene)


My bad, Here's a somewhat longer scene (10 secs, is that allowed here?)

Slightly longer (https://www.dropbox.com/s/c8zcvuzakud58lj/taylor.m2ts?dl=0)

On this small fragment, removing pulldown with tsMuxer seemed fine. On the whole DVD, though, I got this strange error (from tsMuxer):

Warning! Source stream contain irregular pulldown marks. Mistiming between original fps and fps/1.25(without pulldown) exceed 100ms.

As always, your collective help is much appreciated!

[Sharc]

It's definitely soft telecined using pulldown flags.
The pulldown pattern is regular, and removing the pulldown flag or leaving the flag alone and apply IVTC via TFM().tdecimate() produces identical results.
There seems to be a hiccup though at the very end of the clip, but I suspect it is just due to the cutting.

[asarian]

It's definitely soft telecined using pulldown flags.
The pulldown pattern is regular, and removing the pulldown flag or leaving the flag alone and apply IVTC via TFM().tdecimate() produces identical results.
There seems to be a hiccup though at the very end of the clip, but I suspect it is just due to the cutting.

Thank you kindly! :)

[johnmeyer]

The video you uploaded is simply 24 fps. No pulldown removal is needed. You can edit and deal with it as 24p. As I suspected in my initial post, it does have the pulldown flag set which means that when you play it with a "pulldown aware" player (including a DVD player) duplicate fields will be add in order to achieve 29.97 fps without changing the playback speed.

[Music Fan]

The video you uploaded is simply 24 fps. No pulldown removal is needed.
Open the video in MediaInfo or Gspot, you will see there is a pulldown flag. And I believe 24p in 720.480 without pulldown flag does not exist on DVD.

[asarian]

Open the video in MediaInfo or Gspot, you will see there is a pulldown flag. And I believe 24p in 720.480 without pulldown flag does not exist on DVD.

Both tsMuxer and eac3to, as well as MediaInfo, say it comes with 3:2 pulldown.

Now I'm confused again, LOL.

[asarian]

Hmm, using this:

FFVideoSource ("f:\video\taylor.m2ts")
TFM().tdecimate()

Yields a very weird fps:

f:\jobs\taylor.avs: 720x480, 8732/455 fps, 572 frames
y4m [info]: 720x480p 1:1 @ 8732/455 fps (cfr)

19.1912 fps, to be precise. That can't be right.

[johnmeyer]

This script:

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\ffms2.dll")
source=FFVideoSource("E:\Documents\Dnload\UNPACK\taylor.m2ts")
return source

yields a 23.976 video. As I said above, your video is encoded as 23.976, so the original film frames are there.

There is no hard telecine.

Here's the snapshot of the result of that script, playing in VirtualDub:

http://i177.photobucket.com/albums/w208/johnmeyer/Film%2023.976_zpsr36q76n6.png

Note how the File Information dialog in Virtualdub correctly reports 23.976.

[johnmeyer]

BTW, your "weird fps" is more or less what you would expect when you (incorrectly) apply the standard 3:2 TDecimate to 23.976:

23.976 * (24 /30 ) = 19.1808

I'm not sure how you got to the slightly different roundoff, but it doesn't really matter. Just don't do any IVTC, and you'll get the result you want.

[asarian]

This script:

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\ffms2.dll")
source=FFVideoSource("E:\Documents\Dnload\UNPACK\taylor.m2ts")
return source

yields a 23.976 video. As I said above, your video is encoded as 23.976, so the original film frames are there.

There is no hard telecine.

Here's the snapshot of the result of that script, playing in VirtualDub:

http://i177.photobucket.com/albums/w208/johnmeyer/Film%2023.976_zpsr36q76n6.png

Note how the File Information dialog in Virtualdub correctly reports 23.976.


^^ Brilliant stuff here! :) Thank you very much!

[Sharc]

Hmm, using this:

FFVideoSource ("f:\video\taylor.m2ts")
TFM().tdecimate()

Yields a very weird fps:

f:\jobs\taylor.avs: 720x480, 8732/455 fps, 572 frames
y4m [info]: 720x480p 1:1 @ 8732/455 fps (cfr)

19.1912 fps, to be precise. That can't be right.

This just means that your player is not pulldown aware, means it ignores the pulldown flags.
If the player would respect the pulldown flags you would get 23.976fps.

[asarian]

This just means that your player is not pulldown aware, means it ignores the pulldown flags.
If the player would respect the pulldown flags you would get 23.976fps.

There was no player involved yet. :) The '8732/455 fps' was just what x264 reported.

[Sharc]

There was no player involved yet. :) The '8732/455 fps' was just what x264 reported.
Player or decoder (FFVideoSource), whichever ....
I indexed the clip with DGindexNV and decoded with DGsource("...... .dgi").

[asarian]

Player or decoder (FFVideoSource), whichever ....
I indexed the clip with DGindexNV.


Actually, now that you mention DGindexNV, I just noticed frame inconsistencies with FFVideoSource:

SetMemoryMax (768)
SetMTMode (5,4)
FFVideoSource ("f:\jobs\taylor.mkv")
SetMTMode (3)

I wanted to use my old DGDecNV again (lost it since my clean install of Win 10). While I can still download the binaries from Neuron2's site, it would seem the online license generator is gone.

Are there other frame-safe source indexers out there? (for AviSynth 2.6)

[Sharc]

Actually, now that you mention DGindexNV, I just noticed frame inconsistencies with FFVideoSource:

SetMemoryMax (768)
SetMTMode (5,4)
FFVideoSource ("f:\jobs\taylor.mkv")
SetMTMode (3)

I wanted to use my old DGDecNV again (lost it since my clean install of Win 10). While I can still download the binaries from Neuron2's site, it would seem the online license generator is gone.

Are there other frame-safe source indexers out there? (for AviSynth 2.6)
You may want to try LWLibavVideoSource(...).

P.S. The DG.... license generator worked for me recently. Otherwise you may PM Donald Graft.

[asarian]

You may want to try LWLibavVideoSource(...).

P.S. The DG.... license generator worked for me recently. Otherwise you may PM Donald Graft.

Hmm, seems the old url has become a spam domain:

http://neuron2.net/dgdecnv/dgdecnv.html

And a (newer?) site doesn't have the online license tool any more:

http://rationalqm.us/dgdecnv/


EDIT: Nevermind, found it again. :)

[Sharc]

For comparison I produced 2 index files:
- taylor.dgi from your taylor.m2ts source,
- taylor.track_4113.dgi with the pulldown flags stripped by means of tsMuxeR.
Then I play the 2 clips side-by-side in MPC-HC using the script:

a=DGSource("C:\Temp\Scratch\taylor.dgi")
b=DGSource("C:\Temp\Scratch\taylor.track_4113.dgi")
AssumeTFF(a)
AssumeTFF(b)
c1=TFM(a)
c1a=tdecimate(c1)
c2=b
stackhorizontal(c1a,c2)
The 2 clips play identical frame by frame at 23.976fps with the exception of the last frame. But as mentioned the stream as such is 24fps progressive.

[Music Fan]

Removing the pulldown flag with TsMuxer seems more simple to me.

[asarian]

For comparison I produced 2 index files:
- taylor.dgi from your taylor.m2ts source,
- taylor.track_4113.dgi with the pulldown flags stripped by means of tsMuxeR.
Then I play the 2 clips side-by-side in MPC-HC using the script:

a=DGSource("C:\Temp\Scratch\taylor.dgi")
b=DGSource("C:\Temp\Scratch\taylor.track_4113.dgi")
AssumeTFF(a)
AssumeTFF(b)
c1=TFM(a)
c1a=tdecimate(c1)
c2=b
stackhorizontal(c1a,c2)
The 2 clips play identical frame by frame at 23.976fps with the exception of the last frame. But as mentioned the stream as such is 24fps progressive.

That looks pretty conclusive. :) Thx.

[Sharc]

@MusicFan
Yes. But when you re-encode you should let x264 add it back, in order to make it 29.97 (NTSC) compliant - unless you opt for hard telecining.

Edit:
Hmmm.... I just checked x264.exe --fullhelp:
The "--pulldown xxx" option seems to have disappeared.... :scared:

Edit2:
Ooops. It shows up with x264.exe --help, but not with --fullhelp (!)
For 3:2 pulldown flagging the x264 command is --pulldown 32

[asarian]

You may want to try LWLibavVideoSource(...).

P.S. The DG.... license generator worked for me recently. Otherwise you may PM Donald Graft.

Good old DGSource did the trick. : ) (After I found the license generator again). I shall give LWLibavVideoSource a try too.

Still kinda bizarre (even the latest) FFVideoSource still has trouble being frame-stable. But, oh well, it's solved now.

[Sharc]

Good old DGSource did the trick. : ) (After I found the license generator again). I shall give LWLibavVideoSource a try too.

Still kinda bizarre (even the latest) FFVideoSource still has trouble being frame-stable. But, oh well, it's solved now.
I prefer to use DG....NV tools. Too many issues or doubts with other source filters for avisynth, mainly with interlaced sources.

[asarian]

I prefer to use DG....NV tools. Too many issues or doubts with other source filters for avisynth, mainly with interlaced sources.


Wait, isn't DGSource part of DGDecNV?! Like:

LoadPlugin("C:\Program Files (x86)\dgdecnv2051\DGDecodeNV.dll")

SetMemoryMax(768)
SetMTMode(5,4)
DGSource("f:\jobs\Zenon.dgi").ConvertToYV12()

I've alway used DGDecNV like this.

[Sharc]

Yes. I just wrote DG.....NV tools, meaning
DGIndexNV.exe, DGDecodeNV.dll, DGSource()

[asarian]

Yes. I just wrote DG.....NV tools, meaning
DGIndexNV.exe, DGDecodeNV.dll, DGSource()

Ah, I see. :) I misread you. I thought you meant you preferred the DG NV Tools *over* DGSource. :o

But yes, good ol' Neuron2 (wherever he is now) certainly wrote a fine source filter suite!