Problem with animation 2:3 pulldown mixed with 1 progressive 1 interlaced pattern

[shph]

Hello.
I noticed that a plenty of animation DVDs use some sort of special uncommon telecine logic. In some scenes used real 2:3 Pulldown pattern (3 progressive, 2 interlaced), but in some other scenes sometimes pattern changes and it looks like 4 progressive frames 1 interlaced. Sometimes pattern changes to 1 progressive 1 interlaced. In many cases progressive frames in those uncommon patterns have damaged jagged lines.

Usually there is no 2:3 Pulldown tags in those original VOB files.
MediaInfo detect files as:
29.970 (30000/1001) FPS NTSC
Scan type: Interlaced
Scan order: Top Field First

If i manually set input as Progressive and use TFM of VFM filters, i can output to 29.97 and in worst case scenario i can see that in those problematic scenes pattern consists of one normal deinterlaced nice looking frame and one similar original frame with jagged lines.

If i add sRestore - it randomly use those jagged frames instead of normal looking duplicates.

If i manually set input as telecine and use VIVTC or TIVTC - i can also see that sometimes jagged frames may appear.

VIVTC/TIVTC can remove most of those problematic frames, but some amount of jagged frames unfortunately remains. I tried different modes and settings and none of them really helps. Changing Block (x/y) and MI settings to large amounts helps to drop more jagged frames, but same time it adds blending artifacts to other normal frames. So it is not a real solution.

So the questions are:
Why sRestore and VIVTC or TIVTC sometimes select those low quality progressive jagged frames instead of nice looking deinterlaced frames?
Is it some global bug with plugins or with some settings?

Here is source clip example https://drive.google.com/file/d/1gIP...ew?usp=sharing
If use 29.970 framerate problem starts from frame 2342
If use 23.976 framerate problem starts from frame 1873

Input: Progressive


Input: Progressive + TFM


Input: Telecine + TIVTC

[Yomiko]

"4 progressive frames 1 interlaced"
24 fps
Aa Bb Cc Dd -> Aa Bb Bc Cd Dd
12 fps
Aa Aa Cc Cc -> Aa Aa Ac Cc Cc

"1 progressive frames 1 interlaced"
The cut from frame 2342 was directed in 30 fps.

VIVTC/TIVTC issues:
VFM usually doesn't modify fields. TFM usually ignores minor aliasing, so frames even with jagged lines were marked as good ones for field matching.

[Julek]

For 1:1 scene you can freeze:

src = core.lsmas.LWLibavSource(r"D:\Downloads\Alien Nine [CROP].mkv")
ivtc = core.vivtc.VFM(src, 1)
bad_frames = list(range(2342, 2373, 2))
freez = core.std.FreezeFrames(ivtc, first=bad_frames, last=bad_frames, replacement=[i + 1 for i in bad_frames])

[shph]

It is not too realistic to manually check all movie frame by frame and detect sections that use different patterns than normal telecine.
I expected that maybe some autodetection exists?
But i will try to do some deeper frame by frame research and maybe look for some constant pattern logic for those 1:1 parts.

Guess the problem is not "1 progressive frames 1 interlaced" itself. IVTC detects that pattern and visually ok converts things to 23.976.
The problem is in those strange progressive jagged frames already existed in source video. IVTC detects them as good frames but in reality they are bad frames and should be ignored.
It is also unknown if those jagged duplicated frames where placed for some purpose or simply due some mistake during film-to-digital master copy transfer.
I noticed a lot of animation DVDs that have similar artifacts and unusual "variable" telecine pattern. In most cases those artifacts are placed in rare random single frames and not so extremely visible as in example provided here.

[shph]

By the way, i looked for some existing x264 DVD-rips made by other people from the same DVD release, and those rips don't have those jagged frames.
So in theory it is somehow possible to bypass those problematic jagged frames...

[shph]

I got some feedback from authors of DVD rips:

"Used the Yatta software, in which you can detect patterns and enforce them (for the 3:2 sections) as weell as do other manual stuff like freezeframing the cuts where one field is missing and you can also select the 30fps sections so that they don't get decimated, the software generates timecode files for you and the end file is VFR then - 24fps where it was IVTCed and 30fps elsewhere.
It was an awful lot of work to do it that way (besides watching out for 60i fades done on top of the ^ and having to apply debanding in places)"

Yatta software is some very special Avisynth tool https://forum.doom9.org/showthread.php?t=161653
So it is possible to autodetect those variable telecine patterns, but it is not a one click workflow...

P.S. There is also more resent YATTA for VapourSynth (real name: Wobbly) https://forum.doom9.org/showthread.php?t=172496

[Yomiko]

Wobbly is usually not mentioned because we don't believe it works well ;_;

[shph]

I checked Wobbly, and it looks rather complicated manual tool that needs some learning to understand how it works. Not a one click autodetection tool as i expected

I guess in more user friendly way it could be some extended option for VIVTC/TIVTC. Or some special "Variable Telecine" input mode that will autoedect frame patterns and tell to VIVTC/TIVTC/VDecimate how to process them correctly...

[Selur]

Wobbly/Yatta are tools that are meant for folks who have the time and motivation to tweak and know what they are doing, so without a good understanding of what's happening I would not recommend them, but simply live with the normal results.
It's like with most other scripts, they are meant as a usually good enough solution and allows users to not having to know each and every thing and tweak each possible parameter and filter combination.

[Julek]

if you want to automatically freeze when 1:1 you can use Eval:

src = core.lsmas.LWLibavSource(r"D:\Downloads\Alien Nine [CROP].mkv")
ivtc = core.vivtc.VFM(src, 1)

isComb = core.tdm.IsCombed(src)
prev = isComb[:1] + isComb[:-1]
next = isComb[1:] + isComb[-1:]

def freez(n, f, clip):
    comb = f[0].props['_Combed'] + f[1].props['_Combed']
    return clip[n+1] if comb == 2 else clip

from functools import partial
feval = core.std.FrameEval(ivtc, partial(freez, clip=ivtc), prop_src=[prev, next])

[PRAGMA]

Try with my PD2V (PD2V class of pvsfunc): https://github.com/rlaphoenix/pvsfunc

And use this:

from pvsfunc import PD2V, PKernel
from havsfunc import QTGMC

clip = PD2V(r"D:\...\the\file\container\doesnt\matter\trust\me\see\docs.mkv", verbose=True).\
	ceil().\
	deinterlace(kernel=functools.partial(QTGMC, FPSDivisor=2, Preset="Very Slow"), verbose=True).\
	clip

clip.set_output()

This should give you all the information you need about the source, especially information like if its actually a mix of progressive and interlaced or not, and if so, how many are progressive, interlaced, and % of so.
This PD2V script creates an ACCURATE D2V with OPTIMAL settings forced onto it for you, no having to manually index. It then reads the .D2V file directly in python to understand the source better. It then applies fixes in certain scenarios that are caused by d2v_source plugin. Finally, only interlaced frames get thrown into a deinterlacer, meaning all progressive frames are as good as possible.

I hope this truly helps. Any feedback would be great (as an issue if possible as I dont go on here much).

[PRAGMA]

Why sRestore and VIVTC or TIVTC sometimes select those low quality progressive jagged frames instead of nice looking deinterlaced frames?

https://git.io/avoid-tdecimate

VIVTC/TIVTC are of the same, VIVTC is a port so refer to VIVTC as TIVTC the same. TIVTC is basically just an algorithm that by a bit of detections on frames it decides which ones are interlaced, combed (progressive interlaced/burned-in), and progressive. There are a few problems with it highlighted in that git.io link. tl-dr though; Noise (especially in old animation) can be a big problem for TIVTC and can completely throw it off its game on which frames to keep.

[PRAGMA]

As for the screenshots you show, Input: Progressive + TFM is basically just progressive recovering, you actually discarded information here which is why it looks so bad once TFM chews it up.

As for Input: Telecine + TIVTC, it's closer to what you actually want for multiple reasons.

First things first is this looks like the manufacturer chewed it up and spit it out in terms of FPS mangling. It looks as if it was converted from maybe 25.000 FPS (which may have already been a 24->24/1.001 -> 25 conversion) and then converted it to 30000/1001, but I honestly cannot say for sure atm. I would have a better answer for you if you run it through PD2V with verbose enabled.

Either way, the original is most likely 24 or 24000/1001. This error you see where its jaggy, then weirdly super-smoothed yet looks like it has line-doubled effect is actually most likely an editing mistake on the very original file. Family Guy S01-S04 (DVD Volumes 1-3 NTSC) has this very exact issue, so does Simpsons for basically the entire (good part) of the show, and also Spongebob S03 NTSC (but not PAL but it has its own issues).

What is the error happening here? Well, blame Adobe (yet again they still do this day dont properly handle interlaced matieral, especially mixed-scan material). Adobe Premiere 6 for Mac has a nasty nasty bug related to changing of speed and splicing clips that caused it to deinterlace, and reinterlace, and then deinterlace again, and reinterlace on media export in nasty nasty nasty ways. This shows out to me as this issue mostly as of how very very similar it is to SpongeBob S03, and Family Guy nearing S03/S04. Notice how in some frames you can faintly see content from the previous and/or next frame (yeah sometimes both), this is part of that issue.

Sadly, I do not know a fix for that as I'm still tryna hunt a good one down for SpongeBob S03 specifically as SpongeBob doesnt have it too bad (but its noticeable for sure), but Family Guy has it on almost every scene and 2x worse. Sad situation really.

Hope this helps.

EDIT: P.S. The reason you most likely see this issue only sometimes is because it may only be happening on scenes that went through some post-production edits like Pan & Scan, Zoom, or splicing/trimming.

[PRAGMA]

Here's my test results on the file you provided in the OP.

Your gif vs. Mine via PD2V (module of pvsfunc):



If you notice, there's no jagged lines, but there is a bit of "frozen" frames, this is essentially the result of manually freezing to replace the jagged frames like someone else suggested, but instead, it isnt needing to be manually done. Yet another benefit to properly reading the D2V file and applying manual fixes to d2v_source and such. This is a perfect example of why I made pvsfunc and PD2V, to solve stupid crap like this on DVDs, by researching interlacing, deinterlacing, frame matching, combing, etc. for ages.

Notice as well just how better quality the edges are, you don't see the weird blur double line shadow thing here (not sure exactly why). This did use QTGMC, which probably did something to help us here as well.

Here's my script:

from pvsfunc import PD2V
from havsfunc import QTGMC
import functools

clip = PD2V(r"C:\Users\redacted\Downloads\Alien Nine [CROP].mkv", verbose=True).\
    ceil().\
    deinterlace(kernel=functools.partial(QTGMC, FPSDivisor=2, Preset="Very Slow"), verbose=True).\
    clip

Finally, look at the Verbose information text provided by PD2V. You can see it lists various information like progressive and interlaced count and % information, pulldown information (there was NONE in this clip you provided, no progressive information). You can also see it is NOT mixed scan type, only Interlaced, as VFR is False, meaning its only one scan type as DVDs must be mixed-scan-type to be able to have a frame-rate other than NTSC/PAL to switch between.

It's returning 720x480 30000/1001 yuv420p8. You may return 24000/1001 instead by replacing the ceil() line with floor() instead. Look into pd2v docs for information on what these both do, and when you shouldnt use it (depends on source basically). For this source, I would honestly recommend you to stick with ceil() because using floor() whenever there's even a slight risk of dodgy interlacing/scan going on, will mean it MAY desync with the audio. For example with Family Guy NTSC Volume 1 Disc 1 Episode 1 there's a scene where Peter drinks with his friends on a couch, that scene has mangled interlacing (burned in as progressive, so combing) which results in the scene actually being shorter than it was intended to be, but when played back its the right FPS for the audio, meaning the audio probably got mangled too. This means if you use floor() here, there will be LESS frames in that scene than intended, so on-playback it will end the scene before the audio ends, so the rest of the video is offset by a bit and delayed. The more this sort of thing happens the more desynced it is by the end of the video.

Because of this though it is easy to see if it has desynced or not at all by just going to the very end and seeing if it **perfectly** matches or not. If it doesnt perfectly match, don't use floor. Regardless, floor() will only be useful if your source is VFR. Meaning if its a DVD, it would need to be fully progressive or mixed scan. Floor() works by basically discarding frames (weaved fields) from interlaced sections to match their frame rate with the progressive sections which will be presumed to be lower-fps, as interlaced sections HAVE to match either NTSC or PAL, whereas progressive sections can be absolutely anything at it's raw thanks to pulldown.

Finally, with all of this said. With my code I provided above, the result looks very nice on playback. You don't notice the missing fields (which is what resulted in the "frozen" frames) and it plays quite smoothly. So if at all anything, this is probably a just fine enough result if you just need an encode done.

P.S. You can always also run a manual 30->24 fps conversion by decimating frames AFTER all the PD2V code has ran to get it as 24000/1001, which for this source I recommend. It does seem to be originally 24 FPS, and as per usual QTGMC isn't great for animation and the "smooth fps" is noticeable.

[PRAGMA]

As a little bonus, here's some frames of the clip you provided, after being ran through the script I showed earlier, and then ran through VSGAN with a 2x model I have (1440x960 output, basically modern anime HD):

(Click the images to get to full lossless image):

[shph]

Thanks for info PRAGMA. That is really great result! VSGAN upscale also looks very nice. Lines are clean and textures are preserved. Guess it only need some small amount of HQ DeRing filter.

By the way i almost sure now that those progressive frames with jagged edges are part of master copy. Probably they are there due some editing mistake. The creators of that cartoon shared early US DVD release here https://archive.org/details/alien-9 That release is lower quality than Japan DVD and have a lot of rainbow chroma artifacts, but same progressive frames have same jagged edges.

[PRAGMA]

Originally Posted by shph

Thanks for info PRAGMA. That is really great result! VSGAN upscale also looks very nice. Lines are clean and textures are preserved. Guess it only need some small amount of HQ DeRing filter.

By the way i almost sure now that those progressive frames with jagged edges are part of master copy. Probably they are there due some editing mistake. The creators of that cartoon shared early US DVD release here https://archive.org/details/alien-9 That release is lower quality than Japan DVD and have a lot of rainbow chroma artifacts, but same progressive frames have same jagged edges.

Yeah as I said, this is almost evidence towards the (kinda common) Adobe Premiere 6 Mac bug. :/

[shph]

PRAGMA, so your function is Windows only because relies on DGIndex D2V project files?
How do you think, is it possible somehow use D2VWitch instead of DGIndex to make it work on macOS too?

[SaurusX]

One of the most hellacious animated titles to deal with interlacing on DVD is the TMNT 2003 series. I'm very inclined to install Vapoursynth to test out Pragma's program and see the results.

[poisondeathray]

Maybe I'm not using it correctly, but PD2V deinterlaces every frame in that clip, even ones where there was a good match. That degrades all the progressive frames. You can see it in the gif too