req: how to reverse 24 fps->PAL->NTSC interlaced. [Archive]

FredThompson

17th July 2017, 22:25

Source is US NTSC DVD of a PAL TV show which was shot on film. Question is how to process this to get back to 24 fps. From what I see, it looks like a sequence of 4 interlaced frames, 2 progressive, repeat. At other times, it seems to be 5 i, 1 p, 4 i, 2 p. However, this is an old, noisy film original with relatively low action and I might not be detecting this properly. FWIW, Hybrid claims it is fieldblended NTSC from PAL.

I haven't done any telecine conversion for many years.

Would someone please direct me towards any discussions which might cover this, especially, how to determine the correct method to get back to the original 24 fps?

Sharc

17th July 2017, 22:48

wonkey_monkey

17th July 2017, 22:50

Just an aside, but if it was a PAL TV show shot on film, then you want to get it back to 25fps, not 24fps.

The pattern suggests it's simply two fields repeated from every original 10 (12/10*25=30). Can you post a sample?

FredThompson

18th July 2017, 04:28

Maybe you're right. Maybe this is as simple as film played at PAL frame rate. If so, the the question becomes how to adjust the audio. I suppose only remuxing would lead to a pitch change so resampling would be necessary? As I said, it's been many years and I don't remember how to do this.

sample is a zipped mp4 from a source VOB. It's too large for attaching here but can be accessed at: https://we.tl/hIyDgUOBrR

Sharc

18th July 2017, 07:32

There are ugly blended fields. IVTC doesn't help. You could try to restore the progressive 25fps original with
TDeint(mode=1) #or a bobber of your choice
srestore(frate=25.0)
Perhaps someone has a better idea.

manolito

18th July 2017, 08:11

Maybe it's because I am not familiar with NTSC at all, but I never understood all that fuss about restoring the original frame rate... :devil:

What I tried with this sample was applying a good bob deinterlacer
yadifmod(mode=1, edeint=tdeint(mode=1, mthreshL=0,mthreshC=0))
followed by a motion interpolated fps conversion to 25 fps. I used johnmeyer's jm_fps (DCT=0, blocksize=16), and I also tried FrameRateConverter by MysteryX with default settings, both results looked pretty good to me.

Just my 2 cents...

Cheers
manolito

manono

18th July 2017, 08:38

sample is a zipped mp4 from a source VOB.
Why? You could have more easily cut an M2V using DGIndex or an MPG using MPEG2CUT2.

There are ugly blended fields. IVTC doesn't help. You could try to restore the progressive 25fps original with
TDeint(mode=1) #or a bobber of your choice
srestore(frate=25.0)
Perhaps someone has a better idea.
Nope, as that's how you unblend an NTSC DVD made from a PAL source. I'd use a better bobber (as you mentioned), either QTGMC or perhaps YadifMod with NNEDI3.

Maybe it's because I am not familiar with NTSC at all, but I never understood all that fuss about restoring the original frame rate... :devil:
The source was originally progressive so why would you want to keep a bit-wasting interlaced and field-blended degraded version? In my opinion, undoing field-blending always improves a video to which that has been done.

wonkey_monkey

18th July 2017, 10:06

A direct un-reencoded cut from the VOB would be better - not that we don't trust you, but we don't know what the re-encoding may have done to it!

FredThompson

18th July 2017, 15:02

I loaded the original VOB into Virtualdub FilterMod then exported to 709 ut then used VidCoder with no processing to get a small file which illustrated the fields.

You guys are correct, I should have posted a more appropriate sample.

Here's an excerpt from the VOB: https://wetransfer.com/downloads/ad5db7491595d1b7c4a6fb11bf26ba2620170718134016/2add49a9899c559553ba11c841f7110c20170718134016/8db9f8 made with TyTool which kept the origianal video and audio GOPs intact.

First step is to recover the film source then I'll start playing with removing some noise and improving chroma and some slight sharpness.

As I said, haven't done this in many, many years. FWIW, I'm finally converting piles of silver DVDs, TiVo MPEG2s and DV into MKV with x264. Thought about x265 but I'm not sure it's a good long-term solution given it's still quite a niche codec. If there's a way to effectively half x264 bitrate which is used by major media distribution, I'd prefer to use that. Quite another topic, though...

johnmeyer

18th July 2017, 16:14

I've watched all the "Foyle's War" episodes and never detected a problem. I streamed some of them, watched others on PBS, and rented a few of the DVDs.

Sharc is correct that the problem has nothing to do with interlacing, because this is not interlaced. Instead, as he said, you have field blending. If you put the video into a script that simply has one line, bob(), you will see that some of the fields are blended. SRestore is the function that is designed to attempt to undo this problem.

Sharc's script works quite well, but you will be left with a 25 fps (PAL speed) progressive video. If you want it to play at the original film speed, then the simplest thing is to use either AssumeFPS(24) or AssumeFPS(24.0 * 1000.0 / 1001.0) at the end of the script. I can't remember if you then have to stretch the sound, or not. You'll just have to see. I also don't know if current day DVD conversions just speed up the 24 fps sound to get it to play at 25 fps, or whether they now do pitch correction. Ideally you'd want to undo whatever might have been done, but you don't know how it was done, and the pitch change is so small, you can't tell just by listening, unless you knew the actors really well. I'd just leave it as is.

hello_hello

18th July 2017, 16:33

It looks like field blended PAL to NTSC to me. Are you sure it was originally 24fps and not 25fps? If it was intended for broadcast (rather than cinema) it was probably shot at 25fps, even if it is "film".
Hybrid is probably correct. I didn't know any GUI's could detect field blending properly. Clever Hybrid.

I borrowed the following idea from an old post (https://forum.doom9.org/showthread.php?t=97447) and turned it into a simple function (it restores to 25fps). Experiment with "Select" values of 1, 2, 3 or 4 for the best blend removal. It works well as long as the blend pattern doesn't change. You could probably use Bob() instead of Yadif(mode=1) but the field blending process invariable creates interlace-like problems.

Function FixBlend(clip orig, int "Select")
{
Select = default(Select, 3)
SelectCheck = (Select == 1) ? (1) : (Select == 2) ? (2) : (Select == 3) ? (3) : (Select == 4) ? (4) : (5)
assert(SelectCheck != 5, """ FixBlend: "Select" value must be 1, 2, 3 or 4 """)
Selected = (Select - 1)
orig.Yadif(mode=1)
changefps(25*4, linear=false)
selectevery(4, Selected)
return last
}

I use the above script for experimenting and the one below for the actual encoding. It's exactly the same, except QTGMC is deinterlacing instead of Yadif.

Function FixBlendQT(clip orig, int "Select")
{
Select = default(Select, 3)
SelectCheck = (Select == 1) ? (1) : (Select == 2) ? (2) : (Select == 3) ? (3) : (Select == 4) ? (4) : (5)
assert(SelectCheck != 5, """ FixBlend: "Select" value must be 1, 2, 3 or 4 """)
Selected = (Select - 1)
orig.QTGMC()
changefps(25*4, linear=false)
selectevery(4, Selected)
return last
}

This might offer a speed/quality compromise if QTGMC de-interlacing is too slow:

FixBlend()
QTGMC(InputType=1, Preset="Fast")

If the blend pattern changes:

QTGMC() # or Yadif(mode=1) or some other bobber
Srestore()

Hybrid would probably use Srestore (http://avisynth.nl/index.php/Srestore) for that sort of thing. I guess you'd have to follow it with AssumeFPS(24) if it was originally 24fps, but it probably wasn't, otherwise logically it would have been converted directly from 24fps to NTSC using 3:2 pulldown.

wonkey_monkey

18th July 2017, 18:37

Foyle's War, being a broadcast TV drama, would definitely have been shot and originally broadcast at 25fps. The pattern indicates this too.

johnmeyer

18th July 2017, 19:49

Foyle's War, being a broadcast TV drama, would definitely have been shot and originally broadcast at 25fps. The pattern indicates this too.Then ignore the lines in my post concerning AssumeFPS. Not needed. The 25 fps result from Sharc's TDeint + SRestore script looked pretty darned good.

SaurusX

18th July 2017, 20:50

The easiest way to undo the effect and return the clip to 25fps progressive is the following:

AnimeIVTC(mode=2,bbob=4,normconv=true)

It does the QTGMC and srestore within the script. It's essentially Sharc's suggestion rolled into one step.

manolito

19th July 2017, 09:38

Yes, the new unprocessed VOB sample clearly shows that it is field blended, and indeed Sharc's Srestore script gives very good results.

That is, if I use the old Srestore version 2.7e (the last original MOMonster version). Using the latest version 2.78 from 2017 I do not get any error messages or crashes, but the result looks as if there was no blend removal at all. Very strange...

I should mention that I have to use the old masktools2 version 2.0a48 because my Non-SSE2 CPU does not support newer versions. But this does not explain the miserable failure of the latest Srestore script...

You can compare the results for yourself here:
https://we.tl/i641PM89Dl

Cheers
manolito

hello_hello

19th July 2017, 12:34

Maybe the newer Srestore is more sensitive to frames being requested out of order by other filters. What happens if you try Srestore(cache=10)?
That requires TIVTC for the RequestLinear function.

For me, Srestore 2.78 works fine with the sample, at least with plain old Yadif deinterlacing.

Maybe try AssumeFPS(60000,1001) after the de-interlacing in case the frame rate's a tad off. I'm just guessing, although thinking about it, the output was 25fps so that's probably not the problem.

Do you have GrunT?

manono

19th July 2017, 20:21

but the result looks as if there was no blend removal at all. Very strange...
Is it possible you got the field order wrong? I didn't get the more recent sample, but for the original one I had to add an 'AssumeTFF' before the bobbing and blend removal to make it work properly.

And TDeint isn't a very good bobber to use for this as the aliasing it creates is all to obvious.

johnmeyer

19th July 2017, 21:51

And TDeint isn't a very good bobber to use for this as the aliasing it creates is all to obvious.I've been pretty sloppy about choosing bobbers, but I probably shouldn't be. Is there one you would recommend for this particular situation?

StainlessS

19th July 2017, 22:28

@Hello_hello,

SelectCheck = (Select == 1) ? (1) : (Select == 2) ? (2) : (Select == 3) ? (3) : (Select == 4) ? (4) : (5)
assert(SelectCheck != 5, """ FixBlend: "Select" value must be 1, 2, 3 or 4 """)

Above is a little bit 'wordy', below works just fine and somewhat shorter.

assert(1 <= Select <= 4, """ FixBlend: "Select" value must be 1, 2, 3 or 4 """) :helpful:

EDIT: Above '1 <= Select <= 4' is undocumented feature, most would use '1 <= Select && Select <= 4'.

Sharc

19th July 2017, 22:45

I've been pretty sloppy about choosing bobbers, but I probably shouldn't be. Is there one you would recommend for this particular situation?
QTGMC would always be a hot candidate, I think.
(I've been reluctant with recommending it to prevent that the thread is getting flooded with posts how to make QTGMC work, as it's not quite ghetto ..... ;) )

johnmeyer

19th July 2017, 22:59

QTGMC would always be a hot candidate, I think.
(I've been reluctant with recommending it to prevent that the thread is getting flooded with posts how to make QTGMC work, as it's not quite ghetto ..... ;) )QTGMC would work, of course. I don't want to get too tied up in what words mean, but to me, QTGMC has so many other functions in it, that it is hard to view it as a "bobber." It certainly is a deinterlacer and is, for most people, their "go to" plugin for that chore.

So, I have no argument about using it, but somehow thought manono was hinting at some other "simple" bobber, but maybe I misunderstood.

Here is the list of bobbers, as shown in the AVISynth wiki and while QTGMC is shown, so are a lot of much simpler plugins:

BlendBob
DGBob
Decomb
EEDI2
EEDI3
FieldHint
IBob
KernelDeint
LeakKernelDeint
nnedi3
nnedi3ocl
QTGMC
SangNom2
SmoothDeinterlace
TDeint
TelecideHints
TempGaussMC
Yadif
yadifmod
yadifmod2

Sharc

19th July 2017, 23:18

Yeah, there are many.....
I only tried yadif and yadifmod2 as "simple" alternatives but found TDeint visually better. Jaggies (aliasing) could possibly be reduced with TDeint(type=1) or TDeint(type=3).

hello_hello

20th July 2017, 00:04

@Hello_hello,
Above is a little bit 'wordy', below works just fine and somewhat shorter.
EDIT: Above '1 <= Select <= 4' is undocumented feature, most would use '1 <= Select && Select <= 4'.

Yes, that's a bit neater. Cheers.

manono

20th July 2017, 00:19

Is there one you would recommend for this particular situation?
As sharc mentioned, it's hard to go wrong with QTGMC. And earlier I mentioned my two 'go to' bobbers:

I'd use a better bobber (as you mentioned), either QTGMC or perhaps YadifMod with NNEDI3.

I haven't tested either with this source, though. I downloaded manolito's results and the 'bad' one had the earmarks of wrong field order so I asked if that could be the problem. But both were fairly aliased.

johnmeyer

20th July 2017, 00:53

As sharc mentioned, it's hard to go wrong with QTGMC. And earlier I mentioned my two 'go to' bobbers.OK, so YadifMod as the other alternative to QTGMC. Thanks.

manono

20th July 2017, 04:59

Sure. NNEDI3 is used in YadifMod like this:

YadifMod(mode=1, edeint=nnedi3(field=-2))

I test it alone first, to make sure the field order is good, and then add in SRestore. I use it a lot these days with my mostly crappy sources, as it's faster than QTGMC used with good settings.

manolito

20th July 2017, 06:18

Is it possible you got the field order wrong? I didn't get the more recent sample, but for the original one I had to add an 'AssumeTFF' before the bobbing and blend removal to make it work properly.

And TDeint isn't a very good bobber to use for this as the aliasing it creates is all to obvious.

As the bobber I used this one:
yadifmod(mode=1, edeint=tdeint(mode=1, mthreshL=0,mthreshC=0))
This was recommended by tritical himself a few years ago when I asked which deinterlacer would give me the "most bang for the buck". And to me the results look pretty good.

As for the field order of course I did have "AssumeTFF()" before the bobber in both conversions. I made it a habit a long time ago to always explicitly specify the field order in my scripts because of the stupid AviSynth default BFF.

The aliasing you noticed could have come from the resizing to PAL resolution after the Srestore command (I used Spline36Resize).

I still have this strong feeling that the latest Srestore script does not play well together with the older MaskTools2 version 2.0a48. I cannot test this on my machine because newer MaskTools2 versions won't run here (no SSE2 capable CPU). What puzzles me is that I do not get any errors during the conversion even when the result is clearly borked.

Cheers
manolito

hello_hello

20th July 2017, 12:29

manolito,
I tried combinations of SRestore versions 2.7e/2.78 and MaskTools2 2.0a48/2.2.10. I didn't add AssumeTFF() because I checked with Info() and Avisynth was already declaring the vob file to be TFF. Isn't the BFF assumption a PAL thing? I'd have to check, but I had anothert NTSC DVD handy and when I checked it with Info() Avisynth was declaring TFF again.
According to the info here (http://avisynth.nl/index.php/AssumeFrameBased), DGDecode should tell Avisynth the correct filed order and for anything else it'll probably assume BFF. I guess even DGDecode isn't always right because I've had to use AssumeTFF/BFF on occasion, but if frames are in the correct order after de-interlacing I generally don't specify anything. Anyway....

yadifmod(mode=1, edeint=tdeint(mode=1, mthreshL=0,mthreshC=0))
Srestore()

Each combination of Srestore/MaskTools2 seemed to be okay except MaskTools2 2.0a48 and Srestore 2.78. The motion wasn't "jittery" as I think it was for your encode but it wasn't de-blended properly either. The other combinations all missed the first couple of blends but that's to be expected.

I did wonder if the old/new versions of the Average plugin might make a difference. Not enough to test it though. I'm using the new version.

SaurusX

21st July 2017, 16:43

manolito

26th July 2017, 14:09

I did wonder if the old/new versions of the Average plugin might make a difference. Not enough to test it though. I'm using the new version.

Thanks for testing and confirming my findings...

I do use an old version of Average.dll, but it should not matter, because it is only used for double-blend removal:
Average - (only for double-blend-removal, not default)
And I did not specify double-blend removal.

Cheers
manolito

hello_hello

1st August 2017, 17:12

It's usually beneficial to use the dclip parameter on srestore due to its sensitivity to additional blends created by motion-compensating deinterlacers.

MPEG2SOURCE("yourclip")
i=MPEG2SOURCE("yourclip")
QTGMC(Preset="slower",edimode="nnedi3",tr2=1) #mo-comp bobber is OK because of dclip in srestore
srestore(frate=25, speed=-1, cache=10, dclip=i.bob(-0.2,0.6).reduceflicker(strength=1)) #reduceflicker is in the srestore guide and DOES make a difference in some clips

After reading your post I thought I'd give dclip a try, but it seemed to reduce the accuracy of the blend detection just a little. I have no idea why, but I tried cropping the detection clip so the borders wouldn't interfere with the blend detection (as suggested in the help file). I was using an NTSC DVD containing fairly low quality footage as the source (an old BBC series from the 70s).

I'm using Srestore 2.78. If I find the motiovation I'll try again with an older version to see if there's any difference.

Edit: I've deleted most of this post because I was having a stupid and what I posted regarding the dclip option was wrong. See post #33 for a correction.

SaurusX

1st August 2017, 19:56

I'm using Srestore 2.78. If I find the motiovation I'll try again with an older version to see if there's any difference.

I'd definitely be curious as to what you find if you use an older version. I tried "upgrading" to 2.78, but ended up going back to 2.7h after finding the results to be worse. The inner workings of srestore are mysterious.

hello_hello

2nd August 2017, 02:03

It turns out I'm an idiot and I was wrong. The dclip function seems to be working fine for both Srestore 2.7h and Srestore 2.78.
I'm glad I decided to also test an older version of the script otherwise I mightn't have realised it was only my brain that wasn't working properly. Full disclosure:

The script I was really using was the following one, but I removed the Trim() from the second line when I added it to my previous post because I thought it was irrelevant and might confuse the issue.

QTGMC()
trim(52261, 88413).SRestore(thresh=44, speed=1, cache=10)
crop(16, 6, -18, -6)
Spline36Resize(624,468)
GradFun3()
trim(5500, 5999)

When I added the detection clip to the script I should have added the same Trim() to the detection clip:

A=Last.trim(52261, 88413).Yadif(mode=1).crop(16, 8, -16, -8)
QTGMC()
trim(52261, 88413).SRestore(thresh=44, speed=1, cache=10, dclip=A)

I thought I had originally, but when I started re-tested with different de-interlacing for the detection clip I seem to have forgotten to add Trim() to the first line as above, and the end result was the detection clip wasn't from the same section of video as the clip being deblended. Maybe they were almost in sync (blend-wise) because the result probably should have been a lot worse. It's a pity it wasn't, because then the penny probably would've dropped.

Now I've discovered the cause I can at least confirm Srestore 2.7h and Srestore 2.78 produce identical results for the scripts above.
I've only found one instance so far where the detection clip changed the choice of frame, and that was a single frame with some fast motion in a small area of an otherwise static picture and either choice probably could be the correct one. Other than that, the result seems to be much the same with or without a detection clip.... so far. :)