NTSC <-> PAL frame rate conversions with avisynth

[d2mp]

NTSC <-> PAL frame-rate conversion: What is a good way ?

This as been a subject I’ve put particular focus when developing the D2MP tool (http://d2mp.planetaclix.pt)

After some time using these algorithms, I’ve decided to post them and ask guidance to the forum about probable ways of improving them.

Notes:
fps = frames per second;
PAL = 25 fps
NTSC = 29.97 fps (“TRUE” and “TELECINED”)
FILM = 23.976 fps

D2mp uses avisynth scripts to feed QUENC / HCENC / CCE for video encoding.
D2mp uses avisynth scripts to feed FFMPEG for audio encoding.


1. FILM to PAL Conversion:

Converting a 23.976 fps video stream to 25 fps is done trough a simple speedup of the video playback – this is what is usually done by the movie industry when converting a DVD Movie to the PAL European format.

For this effect I’m using the following avisynth command for video streams:

assumeFPS(25)

and since the playback is speeded by about 4% (25/23.976) the audio must be stretched using the following avisynth command:

timestretch(tempo=25.0 * 100 / 23.976)



2. NTSC to PAL conversion:

This one can be tricky.
NTSC 29.97 sources can be (what I call) “pure” NTSC – with really 29.97 frames per second - or they can be previously originally captured at 23.976 fps and converted to 29.97 – what usually happens to material converted from cinema movies (FILM) – by a process usually called TELECINE.


2a. TELECINED NTSC to PAL conversion:
These sources came from original 23.976 fps material usually converted to 29.97 fps repeating 1 video frame (or 2 half-frames) in each group of 5 video frames (there are other FILM-NTSC convertion techniques but I will only consider this one).

The 1st step is to detect if in each group of 5 there is a duplicated frame, and which is the duplicated one.
This analysis can be done by means of an avisynth script which scans the frame differences in a video stream and generates a log file that can then be analyzed.

The 2nd step is to remove these duplicated video frames and so recovering the original 23.976 fps – a process usually called Inverse TELECINE or IVTC. This can be easly done with:

Telecide(order=0,guide=1, post=0)
Decimate(cycle=5,mode=2,quality=3)

The 3rd step will be to convert the now 23.976 fps video and audio material to PAL as discussed in 1.



2b. “TRUE” NTSC to PAL conversion:
Since there is an almost 20% diference between the number of fps, slowing down the playback time to maintain the same original NTSC frames will not be a good approach.
I’m currently using the following avisynth command to adjust the video stream:

convertFPS(25)

The video will be converted blending some frames together to assure the frame rate conversion. There are some noticeable artifacts in the output video stream when playing back some slow moving or slow panning scenes.
This command will not change the stream playback time duration and so the audio stream will not need to be adjusted.



3. PAL to NTSC Conversion:

This one is suposely easier and I’ve used the following avisynth command to convert the video stream:

convertFPS(29.97)

This command will not change the stream playback time duration and so the audio stream will not need to be adjusted.
I’ve decided NOT to speedup the movie from 25 to 29.97 fps since there is a an almost 20% difference which will be very noticeable and not nice to see.
There also some PAL TELECINED sources but I’ve noticed that the above is enough to warranty the conversion even if there are some artifacts as in 2b.



Since I’m not a native English speaker I hope these lines were understandable enough.

I will appreciate any comments to eventually improve the frame rate conversion techniques detailed above.


Thank you very much
d2mp
http://d2mp.planetaclix.pt

[um3k]

I like ConvertMCFPS for frame rate conversion. It's not very fast, but it give much better results than convertFPS (and more compressible, too!). There might be something newer and better (or at least faster), but I don't know about it.

[ajk]

Indeed, give the motion compensated methods (key words for forum search: alchemist, salfps3, mvflowfps2 etc.) a try. They can produce terrific results as long as the material permits. Pure, "natural" sources usually work well, but things like music videos with lots of (cross-)fades, animated content and other things of that nature can really cause problems.

[d2mp]

Indeed, give the motion compensated methods (key words for forum search: alchemist, salfps3, mvflowfps2 etc.) a try. They can produce terrific results as long as the material permits. Pure, "natural" sources usually work well, but things like music videos with lots of (cross-)fades, animated content and other things of that nature can really cause problems.

Thanks for the suggestion.
I assume you are referring for the TRUE NTSC to PAL and PAL to NTSC conversions ?

Regards
D2mp

[ajk]

@d2mp

Yes, mainly for those (I suppose you could do FILM to PAL that way too but it would probably be a bit over the top).

[Boulder]

3. PAL to NTSC Conversion:

This one is suposely easier and I’ve used the following avisynth command to convert the video stream:

convertFPS(29.97)No need for that if the PAL source is progressive. Just resize to NTSC resolution and use DGPulldown to do a 25->29.97fps pulldown.

[2Bdecided]

I see two major problems with the first post.

1. Talking about "the best" way to do something. Did doom9 change in the new year?! ;)

2. Talking about converting (what will usually be) pure interlaced content without taking any account of the interlacing.

The alchemist thread...
http://forum.doom9.org/showthread.php?t=113256
...gives more useful scripts for such content.

Cheers,
David.

[d2mp]

2. Talking about converting (what will usually be) pure interlaced content without taking any account of the interlacing.



Yes. That's true. To simplify the post I've omited the (rare) interlaced material processing scheme.
Just considering the frame rate convertion in this post.
But since you have (and very well) talked about this issue, I believe the simplest aproach for interlaced material is:
a) convert framerate (as explaining in the 1st message);
b) de-interlace;
c) resize to output resolution;
d) re-interlace;
Anyway I will create a separate thread just with the de-interlace / interlace algorithms I'm using in d2mp.


Thank you

[2Bdecided]

I believe the simplest aproach for interlaced material is:
a) convert framerate (as explaining in the 1st message);
b) de-interlace;
c) resize to output resolution;
d) re-interlace;
Quite simple, but I think quite terrible too! "b) de-interlace" should come first.

Given all the excellent threads about this topic, why not pick one of those methods to use?

Cheers,
David.

[d2mp]

OK David. Thank you for your "kind" words.
In fact, looking at the way I've wrote my previous message it sounds awful.
Even being out of the initially planned scope for this thread, let me try to explain better with my limited English:

For the proposes of d2mp program I've found that interlaced material will only be found when converting camcorder material or some DVD-recorder TV captured material.
(for DVD to DVD format conversion there are other tools available).
In fact D2mp pretends to author Dvds using Avis.
So I must say there is around 1% or 2% of interlaced material to be used.
Anyway, I believe these 1% or 2% also deserve to be (tentatively) correctly processed.
I DO NOT want to "de-interlace".
If I have an interlaced AVI, I want to output an interlaced MGP/DVD title.
If I have a progressive AVI I want to output a progressive MPG/DVD title.

So, after detecting an interlaced source material, I try to detect the first field (top or bottom) - there are several ways of doing this explained in the forum.
Then, the fields will be separated (using separatefields).
After all the resizing to DVD standards and adittional processing the fields will be rejoined again (using weave).

I believe that the interlace processing is correct.
What I may doubt is the way frame rate conversion is done between TRUE NTSC and PAL.

But, why not test-it yourself ?
http://d2mp.planetaclix.pt/software/d2mp_install.exe
followed by
http://d2mp.planetaclix.pt/software/d2mp_update.exe

Thanks a lot for your time
Regards

[Boulder]

Do not simply separate the fields, resize and weave. You get vertical misalignment that way. You should use a (smart) bob, resize and then reinterlace.

[d2mp]

Do not simply separate the fields, resize and weave. You get vertical misalignment that way. You should use a (smart) bob, resize and then reinterlace.

Yes. Good idea. I will try that. Thank you.

[Mug Funky]

if you're interested in distinguishing between 3:2 material and video or motion-graphic material in NTSC, you might want to search for the NTSCtools thread.

i'm not sure what the current posted state of it is, but it's as close to the alchemist as i've been able to get within avisynth. only major difference apart from motion-compensation quality is that the alchemist will handle 3:2 v interlaced material within the 1 frame, whereas mine will switch the whole frame between the 3 main types.

it's quite tweakable if you know what's going on. unfortunately i haven't touched it in a year and forget what most of it is doing :) not much call for standards-conversion in my new job, thankfully.

[Floatingshed]

I've checked the NTSCtools thread but there is no sign of a version later than 0.93, there is mention of ver 1.0, did it get this far?
Thanks.

[tebasuna51]

The audio conversion:
timestretch(tempo=25.0 * 100 / 23.976)

or maybe better:
timestretch(tempo=25.0 * 100 / (24000/1001) ) =
timestretch(tempo=100.1 / 0.96 )

preserve the pitch and work with stereo audio, but there are phase shifts problems with multichannel, read this post (http://forum.doom9.org/showthread.php?t=143681) about TimeStretch and multichannel audio

You can use the IanB method:
AssumeSampleRate((AudioRate()*1001+480)/960).SSRC(AudioRate())

This method change the audio pitch but also is "usually done by the movie industry when converting a DVD Movie to the PAL European format."

[DMD]

Good morning
(Sorry for the mistakes I'm using the translator)

I've been trying for days to make the conversion from NTSC to PAL by adopting the method of inverse telecine, using VirtualDub, and then reload the clip 23.976fps thus obtained and setting the frame rate to 25 fps, thus obtaining the clips at 25fps without blending , but with no jerky movements fluids.

To get a better flow, I used an old script used to Motionflow, this one flows better but still not enough and there are still small artifacts.

The script I used is this:

multinum=2
multiden=1
mode=2
spar=0
pel=1
blkh=16
blkv=16
ffdShow_source()
super=MSuper(pel=pel,hpad=blkh, vpad=blkv, levels=4)
backward_vec1=MAnalyse(super, isb=true, blksize=blkh, blksizev=blkv, searchparam=spar, plevel=2, levels=4)
forward_vec1=MAnalyse(super, isb=false, blksize=blkh, blksizev=blkv, searchparam=spar, plevel=2, levels=4)
MBlockFps(super, backward_vec1, forward_vec1, num=FramerateNumerator(last)*multinum, den=FramerateDenominator(last)*multiden, mode=mode)



I also tried the Frame Rate Conversion Filter with processing times crazy.
It has an exceptional fluidity, but sometimes there are blocks of pixels that create artifacts.
http://www.compression.ru/video/frame_rate_conversion/index_en_msu.html

This is the script suggested, I used a value of 2 for less artifacts.

Could anyone give me any suggestions?

Cordially thank

[wonkey_monkey]

Have you tried MFlowFPS instead of MBlockFPS? My understanding was that Flow is better.

David

[Didée]

@ David: More often than not, I like MBlock more than MFlow. The pity is that MBlock doesn't allow block overlapping ... which requires to fiddle it together in other ways, making things slow again. (I like combing 4 different MBlocks, performed on 4 different rotated positions.) :)


[NTSC] ... [[inverse telecine] ... then reload the clip 23.976fps thus obtained and setting the frame rate to 25 fps, thus obtaining the clips at 25fps without blending , but with no jerky movements fluids.
This description is not clear. You have 23.976 fps, then you are "setting the framerate to 25 fps". What? How? Unclear.

After that you post a script that does framerate doubling. How is that connected to the process you tried to describe? Is it that you are doubling the 23976 to 2*23976, then decimating to 25000. Or what, or how?

In the end, there are (IMO) three golden rules about motion-flowing 24fps to 25fps:

1) Don't

2) Do

3) It.

Problem is that the deal is about the long phase of one full second. With a direct conversion 24 > 25, there is only one original frame per second. All remaining frames are interpolated. (Building up for half a second, then declining for half a second again.)
With the 24 > 48 > decimate route, there will be 13 original frames, 12 interpolated, 13 original, 12 interpolated, ...

With either route there will be long sequences of interpolated frames without any original frames. The interpolated frames are prone to have artifacts. Just for ease of arguing lets say the interpolated frames are "error".

Direct flow conversion: One good frame, the error is building up for half a second, then error is decreasing for half a second again. Then comes another good frame. Then half a second error+, half a second error-, one good frame.

Double+decimate: 13 good frames, 12 error frames, 13 good frames, 12 error frames, etc.


It doesn't matter how you turn the coin. 24 > 25 conversion is a beast. There is no good way.

Except for speedup. Speedup is the absolutely preferred way for 24 > 25 conversion. (As long as you can deal with the audio speedup ... not overly critical for general content, but if it's explicitely about "music", it might become a problem indeed.)

[DMD]

Maybe I'm wrong,
"inverse telecine" is obtained by the removal of two frame "not real" every 5, which were added during the Telecine Pull Down,

This is done through VirtualDub Video> Filters Add ... select the IVTC filter go in "Field assembly mode" select "Reduce frame rate"

Getting automatically 23.976fps progressive, then this clip is brought to 25fps, again using VirtualDub.

[Didée]

The IVTC process to 23.976 is a standard process. No need to explain.

The unclear part is "then this clip is brought to 25fps, again using VirtualDub".

How do you get from 23.976 to 25 fps? And how comes the posted Avisynth script into the game?

[manono]

And how comes the posted Avisynth script into the game?
Exactly. This is the AviSynth Forum. The simplest and best answer lies in doing it correctly in an AviSynth script (and not that frame interpolation nonsense). If you want to discuss inferior VDub solutions, do it that forum.

Ordinarily, assuming it's film-based, you IVTC it if necessary, followed by speeding it up. Or, if for DVD, later on after encoding as progressive 23.976fps you could apply DGPulldown for 23.976->25fps, keeping the original speed and audio.

Or, if the source is a DVD, since PAL DVD players already play NTSC sources, do nothing at all and just enjoy the movie.