Pulldown of 25 fps to NTSC

[mrslacker]

Originally posted by Paulcat
Basically then I would have a smaller VOB at the end of it all, but is that the only difference?
http://forum.doom9.org/showthread.php?s=&postid=603872#post603872

Also check the last couple pages of the thread referenced at the beginning of this thread.

[daveidmx]

@neuron2

Sent you a link via PM.

I'm still working on it, so i'll send you some updates as they come.

In the test module, you can change the calling parameters for pdf_init().

You can also switch the two lines just inside that for() loop:

//frame = i;
frame = i + order_table[i % 13];

to test out-of-order frame arrival.

Right now its set for 24->29.97, and fix drop cycle by field.

Also, for testing, in the .H I have the drop cycle set to 30 instead of 1000. (So I suppose its 24->29 then.)

I haven't yet adjusted the leading or implimented the auto-leading, but that's what i'm working on now.

And I have to tweak the initial conditions a bit.

Could I get ahold of one of your test samples?

Crap. OK, fixed that syntax error.

[jptheripper]

thanx for answering the questions.

so steps are

1. resize video to 720x480 25fps
2. run dgpulldown on m2v
3. remux
4. author new dvd.

nice little program you got there

[Sir Didymus]

Originally posted by neuron2
scharfis_brain is correct. You cannot usefully apply pulldown to interlaced source. You can deinterlace it first to make it progressive, however.

You totally lost me. What is a "specular approach"?

I see. Didn't want to talk about silly things; I was referring to the possibility of doing the reversal, for producing a PAL title, starting from a NTSC one...

[scharfis_brain]

Originally posted by mrslacker
It sounds like these 18/20fps silent films were telecined to 29.97. Read this (http://www.doom9.org/ivtc-tut.htm#IVTCForNTSC) little explaination though.

This is a joke, isn't it?

actually I do know, what telecining is.
It is one of my time killers to find out strange telecining patterns to re-progressivize them (eg. 24.45 fps into 59.94 fps).

Talking about telecing here is nonsense,
cause we want to try to AVOID telecining in the video itself.
the telecine will be made by the pulldownflags which neuron2's tool is putting into the stream!
the video itself is stroed progressive without any telecine!


But ontopic again:

neuron2 and I mean:
encode a video at 20fps without telecining.
so you'll end up with a progressive 20fps mpeg.
then add the pulldown flags.
they ensure the telecine for TV.
(the Standalone DVD-Player is actually doing the telecine)

so my question again.
is it allowed to store a progressive stream with
arbitrary (non-valid) framerate into a
container with pulldownflags to reach a
valid framerate (23.976, 24, 25, 29.97, 30, 50, 59.94, 60) ?

[daveidmx]

right now i can chose any input and output frame rate from the list:

1,2,3,4,5,6,8,10,12,15,20,24,25,30

any of which can be X*0.999 also.

so long as (FPS_IN < FPS_OUT < 3/2*FPS_IN)

In about a half an hour i should be able to handle other odd framerates other than those for input.

PS Is it necessary to output frame rates greater than 30?

[scharfis_brain]

right now i can chose any input and output frame rate from the list:

1,2,3,4,5,6,8,10,12,15,20,24,25,30

you've got that from TMPEGenc? right?

TMPEGenc internally does frame repetition (duplication, three, four, ..., and x-times) to reach a valid framrate of
either 24 or 30 or ... fps.

I think, this is NOT what neuron2 wants to achieve.

[daveidmx]

Not from TMPEG, they are the framerates that are factors of the LCM of 24, 25, and 30. Which is the time interval over which I calculate dropping or extending fields. But that will soon change to allow for more free setting of input frame rate.

[Guest]

Originally posted by scharfis_brain
uhm?!?

the MPEG-2 spec. says, that we have only those framerate to choose of:

23.976
24
25
29.97
30
50
59.94
60



how could 18fps, 20fps or any other arbitrary progressive framerate be encoded then?

who is carrying the framerate flag?
the 'unpulldowned', progressive raw MPEG
or the pulldowned result? I agree with scharfis_brain's observation. The idea of my tool is to just flag existing MPEG2. If the encoded frame rate without pulldown is not one of the above rates, then re-encoding is necessary.

If you are close to one of these rates, you can use traditional ways to bring it to that rate, and then use flags to increase it further.

[Guest]

Originally posted by Paulcat
Basically then I would have a smaller VOB at the end of it all, but is that the only difference? It means that with the same bit budget, you can achieve better quality, as well as being able to encode progressively. And you retain the original progressive frames so that they are available again if you ever need them (DGIndex field operation RAW). And you don't have to muck with the audio.

[mrslacker]

Originally posted by scharfis_brain
This is a joke, isn't it?
...
Talking about telecing here is nonsense,
cause we want to try to AVOID telecining in the video itself.
the telecine will be made by the pulldownflags which neuron2's tool is putting into the stream!
the video itself is stroed progressive without any telecine!
...
neuron2 and I mean:
encode a video at 20fps without telecining.
so you'll end up with a progressive 20fps mpeg.
then add the pulldown flags.
they ensure the telecine for TV.

That's what I said (or meant) - stored progressive 20fps and do the telecine with flags. I'm not sure what I was joking about. I pointed to that particular part in the page by manono to get on the same page. I didn't intend to patronize.

[Guest]

Originally posted by Sir Didymus
I see. Didn't want to talk about silly things; I was referring to the possibility of doing the reversal, for producing a PAL title, starting from a NTSC one... That would be possible only if it were encoded at 25fps and flagged by DGPulldown. If so, you could use DGIndex field operation RAW. :)

[daveidmx]

@neuron2

So should I not worry about source frame rates other than standard?

My feeling was to be universal so whatever input frame rate could be flagged to the desired (standard) output rate.

But that may not be necessary if no encoder would produce, say, 20.5 fps.

So: universal input FPS, or factors-of-600 only?

[Guest]

Originally posted by daveidmx
Not from TMPEG, they are the framerates that are factors of the LCM of 24, 25, and 30. Which is the time interval over which I calculate dropping or extending fields. But that will soon change to allow for more free setting of input frame rate. scharfis_brain is correct. The input frame rate must be one of the legal MPEG2 rates. The output rate must also be one of them. The idea is to map the input frame rate to a new output frame rate by manipulating the flags.

Why would you bother trying to support any input or output rates that are not legal MPEG2 values?

[Guest]

Originally posted by daveidmx
@neuron2

So should I not worry about source frame rates other than standard?

My feeling was to be universal so whatever input frame rate could be flagged to the desired (standard) output rate.

But that may not be necessary if no encoder would produce, say, 20.5 fps.

So: universal input FPS, or factors-of-600 only? Legal rates only of course.

You have to very careful if you are thinking of accepting already flagged streams. They may not have constant base rates (hybrid content), i.e., the flagging may not be consistent throughout. So you can't just strip away the old flagging an replace it with new flagging. You'd have to track the base frame rate. Very difficult.

ANyway, the idea here was to start with a progressive unflagged stream and make it 29.97 via flagging. We should discuss the other possible applications before beginning general designs. IMHO.

These are the conversions that could be supported:

23.976 -> 24, 25, 29.97, or 30
24 -> 25, 29.97, or 30
25 -> 29.97, or 30
50 -> 59.94, or 60
59.94 -> 60

For any weird source files, they would have to first be massaged to one of these input rates.

[daveidmx]

@neuron2

Actually it can still be useful exactly because those other rates aren't compliant, in the same way that 25 FPS is not a valid rate for an NTSC DVD.

If, say, someone has captured a home movie as an AVI at, oh, say 22 FPS. They can encode it with 1:1 progressive frames. But of course that's not a valid rate, so then you can signal the bitstream to play back at 29.97 FPS in order to make a legal DVD.

[mrslacker]

Originally posted by daveidmx
@neuron2If, say, someone has captured a home movie as an AVI at, oh, say 22 FPS. They can encode it with 1:1 progressive frames. But of course that's not a valid rate, so then you can signal the bitstream to play back at 29.97 FPS in order to make a legal DVD.

What encoders let you produce a progressive 22fps mpeg? I know CCE is picky. Besides, why not just convertfps(23.976) on that AVI before encoding?

[Guest]

You cannot encode MPEG2 with a frame rate of 22fps. Are you at all familiar with the MPEG2 specification?

[daveidmx]

Originally posted by neuron2
You have to very careful if you are thinking of accepting already flagged streams. They may not have constant base rates (hybrid content), i.e., the flagging may not be consistent throughout. So you can't just strip away the old flagging an replace it with new flagging. You'd have to track the base frame rate. Very difficult.
Already got that part figured out (implementing it now). The tradeoff is that since we have to parse in file in-order and not in temporal order, the only way to do it suffers a subtle generational loss in synchronization. Therefore it should only be used if the source can't be made pure-progressive.
Originally posted by neuron2
ANyway, the idea here was to start with a progressive unflagged stream and make it 29.97 via flagging. We should discuss the other possible applications before beginning general designs. IMHO.
Sorry if i'm jumping ahead, it just seemed like the logical extension of the code i'd already written.

Obviously, the output rate should always be standard. But I figure the input rate may as well be free, to cover cases like in my last post, since it's not really more complicated.



@mrslacker
Originally posted by mrslacker
What encoders let you produce a progressive 22fps mpeg? I know CCE is picky. Besides, why not just convertfps(23.976) on that AVI before encoding?
Maybe not any will let you. But instead of convertfps(), using assumefps() will let you encode it with 1:1 frames while pretending its a valid rate, and then you re-flag that stream to play properly. Remember, convertfps() basically does exactly what we're trying to avoid.

[mrslacker]

Maybe not any will let you. But instead of convertfps(), using assumefps() will let you encode it with 1:1 frames while pretending its a valid rate, and then you re-flag that stream to play properly. Remember, convertfps() basically does exactly what we're trying to avoid. [/B]

As neuron2 said in a more direct manner than I, none will. However, Your nearest unflagged progressive frame rate would be 23.976. I guess you could encode a fast video and change the frame_rate flag to achieve what you suggest, but I would prefer to "massage" into a standard rate through a small frame rate conversion with avisynth. In that case standard pulldown would follow.

[daveidmx]

Originally posted by neuron2
You cannot encode MPEG2 with a frame rate of 22fps. Are you at all familiar with the MPEG2 specification?

Yes I am. And I'm by no means trying to imply that you or anyone else doesn't. Forgive me.

But remember that only the output must be compliant. Temporary files and in-between steps can pretty much be whatever you need in order to make something work.

Again, I'm certainly not trying to pick a fight. I know it's difficult to converse over a thread like this where you never quite get the intonation of the words, and can't deliver a thought face-to-face. Please be a little patient as we try to discuss this.


@mrslacker
Originally posted by mrslacker
I guess you could encode a fast video and change the frame_rate flag to achieve what you suggest,
Exactly. I'm just suggesting that, for the same reasons some people might want to flag a 25fps stream instead of convertfps()ing it, some might like to do the same with their 22 FPS home movie. No-one _has_ to do it, it's simply an option for those who wish to.

[mrslacker]

Originally posted by daveidmx
Exactly. I'm just suggesting that, for the same reasons some people might want to flag a 25fps stream instead of convertfps()ing it, some might like to do the same with their 22 FPS home movie. No-one _has_ to do it, it's simply an option for those who wish to.

OK. I don't have a copy of the MPEG2 specs on my desk at the moment ;), but wouldn't a change in the frame_rate flag break buffering at the least if no fields were repeated? A change in the opposite direction (speed up) being the more obvious example? Just a thought.

[Guest]

@daveidmx

OK, that will work. We are still limited to an original frame rate [before assumefps()]of 2/3 of the target rate.

EDIT: I'm not so sure. Now I think the VBV will get screwed up, as mrslacker observed. You won't be consuming encoded frames at the correct rate. You're trying to throttle during the display process, which is exactly what caused Inwards's experiment to fail.

[daveidmx]

@neuron2
Yeah, haven't totally thought through the VBV thing yet. Obviously in one direction it will get broke, but maybe the other way will work? I really don't know. Maybe I'll just play around later.

Say, do you have a 25 FPS VOB I can play with over here?

[Guest]

mrslacker pointed out the gotcha first. :)

I'll put up a VOB for you tonight.

[scharfis_brain]

is it possible to DROP a field with the rff-flags?
to make the explantation more simple, I only used integers instead of the 1000/1001 - framerates

goal: telecine 18fps to 60fps
(4:3:3 pattern, combed)

input stream 18 fps progressive (super8)

A B C D E F G H I J K L M N O P Q R


hard progressive (non-combed) telecine to 24 fps (2:1:1 pattern)
(use changefps(24) or selectevery(3,0,0,1,2) then encode with 24fps)

AA B C DD E F GG H I JJ K L MM N O PP Q R


then use the rff-flags to build s 60fields per second interlaced stream out of our progressive telecined 24fps mpeg:

this will work, as I assume:


A A B B C D D E E F G G H H I J J K K ...
a a b c c d d e f f g g h i i j j k l ...


but this is questionable (drops a field from the dupe of the 24fps mpeg)
shifted the pattern by one frame in time, as it WILL be the case on 1000/1001 correction:

A A B B C D D E E F G G H H I J J K K ...
a b b c c d e e f f g H h i i j K k l ...


or is the possible though?

if yes, it should be possible to telecine nearly all arbitrary framerates without a need to encode interlaced.

[Guest]

That appears to be just an instance of massaging to a legal MPEG2 rate first, as we discussed.

[daveidmx]

@neuron2
Posted a new version fixing some bugs.
Renamed leading to tracking.
Should now track sync on streams which already have PD flags, though it'll be jerky and will probably break VBV and is not reccommended for the time being. I'll work on optimizing that tomorrow along with the auto-tracking. Still only valid MPEG frame rates handled.
Yippee!

[Guest]

I've just completed a preliminary GUI version. I'm heading home now to test it a little and then upload it for y'all. Of course, it has ES support.

[jptheripper]

wow, such service

quick question,

once the stream is demuxed to m2v and ac3, reencoded to 720x480, then dgpulldown applieed to new m2v, do you have a prefered remuxing engine?

can the ac3 and dgpulldown'd m2v just be dropped into dvdlab pro?

[tyee]

can the ac3 and dgpulldown'd m2v just be dropped into dvdlab pro?

Sure, I would think so if it handles elementary streams.

tyee

[Guest]

OK, here's the GUI version with ES support. Please let me know if you find any problems. Thank you.

http://neuron2.net/misc/dgpulldown100b1.zip

Yes, I know I need a browse button for the input file entry. And a progress bar. :)

[Guest]

Originally posted by jptheripper
once the stream is demuxed to m2v and ac3, reencoded to 720x480, then dgpulldown applieed to new m2v, do you have a prefered remuxing engine? I use TMPGEnc DVD Author. I prefer it because it is all I have. :)

can the ac3 and dgpulldown'd m2v just be dropped into dvdlab pro? As the ever perceptive tyee points out, of course!

And as our friend daveidmx says...wheee!

[jptheripper]

was just curious ..

as for muxing i recommend muxman, cause its free!

[mrslacker]

I took this opportunity to convert a PAL disc that I have stashed, and... success!

disc->DGIndex->avs(resize)->CCE->DGPulldown->MuxMan

The resulting disc plays perfectly (so far) on my Sony NC665P. Hurray! This is a spectacular solution.

[tyee]

neuron2
Just redid my 10 minute clip (with the .m2v file) and perfection again!! Well done!!

tyee

[Guest]

Great. Thanks for the feedback. As a reward, here's a version with a browse button for the input filename:

http://neuron2.net/misc/dgpulldown100b2.zip

[Guest]

@daveidmx

I'm uploading a test file right now. Give it a few hours or so to complete. It will be on the FTP server named:

25fps_long_sample_for_dgpulldown_test.vob

[Paulcat]

Nice job Don! Shame TMPGEnc doesn't use this method...

How far are you from a program that demuxes, resizes AND does your pulldown?
("It's a JOKE, son")

You know, if only manufacturers of players and dvd's realized that the most money could be made by eliminating regions and playing all formats, we wouldn't have to deal with this bollocks.

p.s. Anyone in the UK know when Green Wing will be out on DVD?

[digidragon]

If the MPEG is progressive, but has an interlaced flag, does the flag have to be set to interlaced using Restream, for example?

I'm thinking of many UK TV programmes, which are shot on video, but are either deinterlaced before broadcast to give them a filmic look, or are shot on progressive video. The broadcast MPEGs, although they look progressive, are flagged as interlaced.

[tyee]

neuron2
Just one question on how the pulldown was chosen. Earlier we talked about your method like this --

AABCDE
ABCCDE

This gave 4 progressive and 2 interlaced, and is how you are currently doing it? Yes?

I asked about doing it this way --

ABBCDE
ABCCDE

which gives one interlaced and 5 progressive frames.
I guess I'm trying to understand why you chose the first method. Isn't it better to have the fewest number of interlaced frames?

I'm probably missing something important but I'm still learning.

tyee

[Xesdeeni]

Originally posted by Paulcat
You know, if only manufacturers of players and dvd's realized that the most money could be made by eliminating regions and playing all formats, we wouldn't have to deal with this bollocks.I don't understand this either. Any progressively encoded stream (24, 25, or 30) should be playable on any player. I've never understood why the flags were necessary. If you know the input rate and the output rate, it seems trivial. It's the lack of this ability that finally got me thinking about this flag gymnastics in the first place.

BTW, cudos to DG!

Xesdeeni

[mrslacker]

At this point, It seems almost trivial to implement the 23.976->29.97 conversion. It would be very nice to have DGPulldown assume the roll as the conventional NTSC pulldown tool as well. You did the challenging 25fps source without a hitch as far as I can tell. I'll test my converted disc on some other players to be sure they all respect the flags. Can I say it now? Good Work!

[Xesdeeni]

So what will progressive DVD players do with this? Will they be able to create the right 60 fps cadence?

Xesdeeni

[Guest]

Originally posted by digidragon
If the MPEG is progressive, but has an interlaced flag, does the flag have to be set to interlaced using Restream, for example? What do you mean by "has an interlaced flag"? You have to be precise if I am to understand you. Are you referring to the progressive_sequence field of the sequence extension, the progressive_frame_bit of the picture coding extension, or what? And what is Restream?

My understanding is that both of the aforementioned syntax elements refer to coded pictures, and not display frames.

I'm thinking of many UK TV programmes, which are shot on video, but are either deinterlaced before broadcast to give them a filmic look, or are shot on progressive video. The broadcast MPEGs, although they look progressive, are flagged as interlaced. I guess I just don't see what you are asking me. Sorry.

[Guest]

Originally posted by tyee
ABBCDE
ABCCDE

which gives one interlaced and 5 progressive frames.
I guess I'm trying to understand why you chose the first method. Isn't it better to have the fewest number of interlaced frames? Why do you think that 3:2 is not done like this:

A A B C D
A B B C D

That would give 4P+1I instead of 3P+2I. It's all about smoothness of the resulting video. Your pattern would stutter much more. The idea is to distribute the extra fields, not bunch them up. There's no point to trying to keep the display frames progressive when you are playing to an interlaced display.

[Wilbert]

@digidragon,

Sometimes progressive content is encoded as interlaced (for no reason). That's probably why they are flagged as interlaced.

@neuron2,

And what is Restream?
:search: :)

With this tool you can change many options of a MPEG2 Elementary Stream without re-encoding.
You can change options like the Sequence Header, matrices, tff-flags, User Data, GOP-flags, pan-scan-info ...
http://shh.sysh.de/restream.html

[Guest]

Originally posted by Xesdeeni
I don't understand this either. Any progressively encoded stream (24, 25, or 30) should be playable on any player. I've never understood why the flags were necessary. It's there to handle hybrid material. The 30fps sections are not flagged and the 24fps sections are flagged. Without some flagging, how would your player know what to do with the different sections?

[Guest]

Originally posted by mrslacker
At this point, It seems almost trivial to implement the 23.976->29.97 conversion. It would be very nice to have DGPulldown assume the roll as the conventional NTSC pulldown tool as well. Yes, the other trivial conversions will be implemented.

I'm going to remove program stream support because it is not that useful anyway and it requires me to redo all the PTS stamps, which I have neglected thus far.

[digidragon]

Originally posted by neuron2
What do you mean by "has an interlaced flag"? You have to be precise if I am to understand you. Are you referring to the progressive_sequence field of the sequence extension, the progressive_frame_bit of the picture coding extension, or what?

I guess I just don't see what you are asking me. Sorry.

I mean that the picture coding extension of the broadcast MPEG is TFF. The progressive sequence extension is not set.

I knocked off the TFF flag, and set the progressive sequence extension using ReStream, re-encoded it to 720x480 25fps, and ran it through pulldown. It authored fine as an NTSC DVD, but on my player it froze several times, and stopped playing prematurely.

So, does the source have to be a true progressive source (i.e. film) to use this method?