Problem with animation 2:3 pulldown mixed with 1 progressive 1 interlaced pattern

[Selur]

@PRAGMA: which model did you use?

[PRAGMA]

Originally Posted by shph

PRAGMA, so your function is Windows only because relies on DGIndex D2V project files?
How do you think, is it possible somehow use D2VWitch instead of DGIndex to make it work on macOS too?

It does support Linux and macOS, just download, install wine, add DGIndex.exe to system path.

I've had tons of inaccuracy and glitchy frame read issues with D2VWitch back in the day, so even on Linux I advise people to use DGIndex. All you need to do is download DGIndex, install Wine, and add the binary for DGIndex.exe to your system PATH environment variable (so not your shell environment variable).

That way the script will find DGIndex, detect you are on Linux, and will call DGIndex with Wine for you.

Btw, If you REALLY REALLY want to use D2VWitch or such (which I cannot personally recommend), then simply provide the input path directly to a .d2v and not an MKV, MPG, MP4, and such. This way it wont try to generate a D2V for you and will use the one provided. But remember that PD2V expects specific D2V settings and may have bad unintended behavior if D2VWitch doesnt generate with those settings. See https://github.com/rlaphoenix/pyd2v/...2v/d2v.py#L219

[PRAGMA]

Originally Posted by poisondeathray

Maybe I'm not using it correctly, but PD2V deinterlaces every frame in that clip, even ones where there was a good match. That degrades all the progressive frames. You can see it in the gif too

You must realize that the entire clip provided is in fact interlaced. There are no progressive frames found within. It may be possible to "recover" progressive frames by using something like TFM, but as mentioned before it doesnt work very well in most scenarios. And even then, the doubt that it could potentially mess up even one frame is enough for me not to use it. I'd prefer to let QTGMC or YADIF have at the original interlaced frames at that point.

Regardless, you CAN still use something like TFM with PD2V. Simply replace the .deinterlace(...) call which has for example QTGMC, with a TFM call. Then add ANOTHER .deinterlace() after that one, and add back in QTGMC.

Everything in PD2V is chainable, but once you add .clip it's no longer chainable as you now have the final clip output.

[PRAGMA]

Originally Posted by Selur

@PRAGMA: which model did you use?

I used the model `2X_DigitalFilmV5_Lite`: https://anonfiles.com/n5X805Nfu9/2X_...ilmV5_Lite_pth

[PRAGMA]

Originally Posted by SaurusX

One of the most hellacious animated titles to deal with interlacing on DVD is the TMNT 2003 series. I'm very inclined to install Vapoursynth to test out Pragma's program and see the results.

I ran the same script I gave OP as an example for Alien Nine on TMNT 2003 Disc 1 (AOTM) and it doesn't seem to have much issues at all.

The entire thing is 30000/1001 Interlaced 100.00%. There is definitely a lot of recoverable frames in here, but as I told poisondeathray I don't like using TFM (but that's just me, you do you).

Regardless, after running it seems fine and totally grand so far. Nothing burnt in or anything (that I've seen so far anyway).



This show seems to be 24fps as I can notice duplicate frames every 1 in 5 frames. Since the interlacing here actually look clean, using .floor() instead of .ceil() might be OK, but you should always test for audio desync.

Since this source is 100.00% Interlaced, it is not VFR. And since it's not VFR .floor() won't actually do anything at all as it can not assume any cycle information. So, we must tell it the cycle information. I checked for you and it seems to be `floor(cycle=5, offsets=[0, 1, 3, 4])` (which is the most common for 24->NTSC, essentially discard every 3rd frame in every 5 frame cycle).

[PRAGMA]

Originally Posted by poisondeathray

Maybe I'm not using it correctly, but PD2V deinterlaces every frame in that clip, even ones where there was a good match. That degrades all the progressive frames. You can see it in the gif too

Ok so a bit of an update on this.
Since it seems a fair amount of people want to use VFM (or just recover progressive frames to be specific) I decided I would add support for this in pvsfunc/PD2V.

Support has started in the new v4.2.0.

There's now a `recover()` method that is just an optimizing wrapper over VFM. I would have made it work like `deinterlace()` where you can supply your own kernel/function, but since I dont know any other alternatives to VFM I didnt bother.

This wrapper deals with the clip and order params (though you can force override still) and lets you change any VFM argument you want still.

Speaking of `deinterlace()` it now supports dealing with clips that had VFM ran on it. Meaning if VFM could not recover a frame then no worries as you can chain on a .deinterlace() and deinterlace that frame (and that frame only, leaving the real progressive and recovered progressive alone).

What this means is that if you like to use TFM, but also like my PD2V setup, then yay just go use TFM via recover() instead of manually calling it.

The verbose options are also optimized between deinterlace() and recover() so that they both still show their information and optimized terminology to as to not confuse.



Examples on TMNT 2003 Disc 1:


Script:

import functools
from pvsfunc import PD2V
from havsfunc import QTGMC

clip = PD2V(r"C:\Users\phoenix\Videos\TMNT\title_t00.mkv", verbose=True).\
    recover(mode=0, verbose=True).\
    deinterlace(kernel=functools.partial(QTGMC, FPSDivisor=2, Preset="Very Slow"), verbose=True).\
    clip

As you can see on the last image is that it isnt always perfect. The 4th image is a scenario of which VFM could not recover a progressive frame but it got passed to QTGMC for deinterlacing

Also, the whole "double-line" thing that I referred to a few days ago now seemingly was caused by VFM/TFM. I say this because when recovered I can notice it at the beginning of this TMNT on edges. So, it would see recovering progressive frames can sometimes cause that when QTGMC doesnt.



As you can see on the left image (default settings TFM call), near the top left you can see some double-line (aka thick line). Whereas on the right image (No TFM, QTGMC Very Slow), you don't, you only see a typical SD-era blurry line.

[shph]

That's interesting. I thought that "double-line" in Alien 9 was a semi-transparent film celluloid (At least it looked very similar to this). But in TMNT 2003 it is really looks like some unusual artifact generated by lack of deinterlacer.

[shph]

Originally Posted by PRAGMA

Btw, If you REALLY REALLY want to use D2VWitch or such (which I cannot personally recommend), then simply provide the input path directly to a .d2v and not an MKV, MPG, MP4, and such. This way it wont try to generate a D2V for you and will use the one provided. But remember that PD2V expects specific D2V settings and may have bad unintended behavior if D2VWitch doesnt generate with those settings. See https://github.com/rlaphoenix/pyd2v/...2v/d2v.py#L219

Ok, so maybe D2VWitch could be modified and improved in future versions to fully support PD2V? I will try to ask developers ...

[SaurusX]

Originally Posted by PRAGMA

As you can see on the last image is that it isnt always perfect.

Thanks for taking a look at this series. I think using ceil() would be the way to go here, because there are plenty of areas that are not 24fps and are either 30fps or 60i. That image with Splinter being extremely aliased is not uncommon, either. It seems to be an artifact of whatever animation software was being used. Part of the frame will look low res like a bob deinterlace, while the rest remains sharp.

[Selur]

I used the model `2X_DigitalFilmV5_Lite

Thanks! Another model for my collection.

Cu Selur

[poisondeathray]

Originally Posted by PRAGMA

You must realize that the entire clip provided is in fact interlaced. There are no progressive frames found within. It may be possible to "recover" progressive frames by using something like TFM, but as mentioned before it doesnt work very well in most scenarios. And even then, the doubt that it could potentially mess up even one frame is enough for me not to use it. I'd prefer to let QTGMC or YADIF have at the original interlaced frames at that point.

The content is 99% all progressive (most animations are) but the type of encoding is interlaced on this DVD (as many DVD's are) - ie. it's encoded as fields, instead of native progressive with repeat field flags (the latter is "soft pulldown"). The d2v conveys encoding type, not necessarily any accurate info about content - the author, DG, will tell you basically the same thing. The content is what is important.

So on DVD's that are encoded interlaced, you'd advocate deinterlacing everything ? On something like that OP's DVD, that potentially degrades >99% frames, for fixing a handful <1%. As good as QTGMC is, you still soften the image and create *new* problems with aliasing and ghosting . Some of the QTGMC frames are almost as good as progressive, but some are much worse.

Since you're adverse to 1 bad frame - I'd argue that you shouldn't deinterlace progressive content at all - because you cause *new* additional problems that were not in the source. But hey it's your choice...










Now, there are some other bad sections like interlaced fades that might not be detected by normal comb detection - but the "bad" sections on this DVD sample is <1%.

Some other DVD's might have some interlaced overlays or titles , a few true interlaced content sections; or on some very badly produced DVD's, you might get something like 90% bad frames (for various reasons). On those latter sorts of scenarios it might make sense to "blindly" QTGMC everything to smooth things over. But it' s nothing that the d2v can tell you about - you have to use your eyes

This d2v method might be more useful on DVD's that are mixed - that have soft pulldown (encoded progressive) and hard pulldown (encoded interlaced) on the same title

Originally Posted by PRAGMA

Ok so a bit of an update on this.
Since it seems a fair amount of people want to use VFM (or just recover progressive frames to be specific) I decided I would add support for this in pvsfunc/PD2V.

Support has started in the new v4.2.0.

Thanks for the updates, it's nice to have options.

But this option is basically the same as field matching (VFM or TFM in vapoursynth) + PP . If combing is not detected , it still allows passthrough of the "bad" frame . The garbage frames in the OP's source are not "combed", in the traditional sense, so they are not detected. A different type of detection is required, or human eye.

Eitherway there are compromises . I appreciate the work you are trying, keep it up

[shph]

Yep, i got same problems when attempt to deinterlace everything (D2VWitch+QTGMC). In other forum Selur also suggested just deinterlace everything, but i didn't like overall final result.
If PD2V deinterlaces every frame in that clip, sure QTGMC will generate new artifacts. When PRAGMA introduced that method i thought that DGIndex+PD2V deinterlace everything in some other "selective" way.

[PRAGMA]

Originally Posted by shph

Ok, so maybe D2VWitch could be modified and improved in future versions to fully support PD2V? I will try to ask developers ...

The sole problem with D2VWitch isn't the incompatibility with PD2V, it's the inaccuracy and reading issues I've personally encountered on various sources (both clean and dodgy scan situations) using a D2V from D2VWitch and d2v_source plugin.

If support for D2VWitch is REALLY wanted, then make an issue in pyd2v project (and not PD2V) and if enough people show their interest I may add in an option to use D2VWitch if DGIndex could not be found on the system.

[PRAGMA]

Originally Posted by SaurusX

Thanks for taking a look at this series. I think using ceil() would be the way to go here, because there are plenty of areas that are not 24fps and are either 30fps or 60i. That image with Splinter being extremely aliased is not uncommon, either. It seems to be an artifact of whatever animation software was being used. Part of the frame will look low res like a bob deinterlace, while the rest remains sharp.

Yes correct. I've noticed this in the intro as well. Again, seems to be caused by the dreaded Adobe bug.

And I would agree with you, it seems for this show a ceil() would be best instead of flooring all the potential 60i post-prod. However, I have noticed a lot of scenes and such that could potentially have been post-prod, doesn't seem to be 60i/30p still. So I would honestly be OK if someone did do a floor() here, since there isnt THAT much 60i, and the benefits of lower file size would outweigh the pros of 10% at most being 60i/30p.

[PRAGMA]

Originally Posted by poisondeathray

The content is 99% all progressive (most animations are) but the type of encoding is interlaced on this DVD (as many DVD's are) - ie. it's encoded as fields, instead of native progressive with repeat field flags (the latter is "soft pulldown"). The d2v conveys encoding type, not necessarily any accurate info about content - the author, DG, will tell you basically the same thing. The content is what is important.

So on DVD's that are encoded interlaced, you'd advocate deinterlacing everything ? On something like that OP's DVD, that potentially degrades >99% frames, for fixing a handful <1%. As good as QTGMC is, you still soften the image and create *new* problems with aliasing and ghosting . Some of the QTGMC frames are almost as good as progressive, but some are much worse.

Since you're adverse to 1 bad frame - I'd argue that you shouldn't deinterlace progressive content at all - because you cause *new* additional problems that were not in the source. But hey it's your choice...

Now, there are some other bad sections like interlaced fades that might not be detected by normal comb detection - but the "bad" sections on this DVD sample is <1%.

Some other DVD's might have some interlaced overlays or titles , a few true interlaced content sections; or on some very badly produced DVD's, you might get something like 90% bad frames (for various reasons). On those latter sorts of scenarios it might make sense to "blindly" QTGMC everything to smooth things over. But it' s nothing that the d2v can tell you about - you have to use your eyes

This d2v method might be more useful on DVD's that are mixed - that have soft pulldown (encoded progressive) and hard pulldown (encoded interlaced) on the same title





Thanks for the updates, it's nice to have options.

But this option is basically the same as field matching (VFM or TFM in vapoursynth) + PP . If combing is not detected , it still allows passthrough of the "bad" frame . The garbage frames in the OP's source are not "combed", in the traditional sense, so they are not detected. A different type of detection is required, or human eye.

Eitherway there are compromises . I appreciate the work you are trying, keep it up

Hi, I was initially saying that I do not like to use TFM as the sources I tended to work with did not fair well at all.

However, as you pointed out this source and TMNT (which both are relatively clean sources) are working quite well with VFM in most frames.

You also pointed out how frames with fade in/out of either the entire frame or just text on-screen and such has problems being frame matched and yes that is true, that is one of the reasons I tended to skip out on TFM in some of my sources as even if I let it passthrough to QTGMC, the differences of the results were noticeable on said source (as in sharpness of lines, purity, and such).

After adding in recover() in v4.2.0 (and a regression fix in v4.2.1) and testing it on TMNT and Alien Nine, I will say I'm impressed with the results.

This is the type of thing I'd love to use for example on Sonic Underground PAL (10-disc set, not the 5-disc set that is WORSE!), as it has similar scan situation to TMNT, except that literally like 90% of every disc can be recovered without a hitch due to the animation being 25fps exactly, so every 2 fields would have made up one full frame without any time-displacement between them! The only times matching could not happen was with post-prod pan&scans, which were full 50i as in every 2nd field was 1 time-unit ahead of the previous, e.g. like Live-action soaps tend to do interlacing in PAL regions (e.g. IT Crowd).

I'm not saying that TFM/VFM shouldn't be used, but I do advise caution on testing and tuning the VFM call as it can be disastrous at times like with TMNT where I highlighted the double-line issue.

An update on said double-line issue, I actually know EXACTLY what the problem is here. Every few frames or so, in a smooth yet linear timeframe, that scene gets blurrier and sharper as it goes on (probably an error during a post-prod pan and scan), and when VFM is field-matching a frame that is e.g. blurrier to the current frame which may be e.g. either sharper or blurrier than the other field its matching with, it causes a noticeable mismatch effect which showcases itself as if the entire frame got sharpness+10 thrown on it. This is the kind of thing that I dont like VFM for, the almost invisible on paper mistakes that can happen.

Regardless I'm glad it's working out for some sources but just needs a little per-source TLC.

[PRAGMA]

Originally Posted by shph

Yep, i got same problems when attempt to deinterlace everything (D2VWitch+QTGMC). In other forum Selur also suggested just deinterlace everything, but i didn't like overall final result.
If PD2V deinterlaces every frame in that clip, sure QTGMC will generate new artifacts. When PRAGMA introduced that method i thought that DGIndex+PD2V deinterlace everything in some other "selective" way.

To be clear, PD2V does NOT do *any* deinterlacing at all.
PD2V is simply a D2V optimized func-chaining system, which is also more tailored specifically to MPEG DVD video files (some assumptions in the code are specifically made assuming that the input is valid for the DVD-Video Spec).

When you use .deinterlace(), you are providing .deinterlace a kernel (a function, e.g. QTGMC) with your OWN arguments (e.g. preset VerySlow, though it handles stuff like order and TFF automatically).

When .deinterlace actually uses the kernel you provided, it will ONLY be used on frames marked as interlaced by the D2V file. Any frame marked as progressive either by the D2V file, or e.g. by recover(), will NOT go through the kernel ever.

This is why chaining .recover().deinterlace() is possible and supported.

Originally Posted by shph

When PRAGMA introduced that method i thought that DGIndex+PD2V deinterlace everything in some other "selective" way.

This is true. It does that. It selectively deinterlaces only frames that should be, as told by the D2V file. Again, I stress about DGIndex over D2VWitch because PD2V (via pyd2v) generates D2V files with very specific settings. If any of the settings on-generation are different then a totally wrong or bad result may happen if used through PD2V.

Reminder that when PD2V loads a D2V, it also applies fixes to the loaded clip that d2v_source plugin returns. There are scenarios in which if a clip is VFR, d2v_source does not correctly load in the source, it loads it at either NTSC or PAL frame rates HARDCODED and NOT dynamic/VFR, it loads as constant only ever.

When PD2V loads the clip, it then fixes the loaded clip to be correct intended FPS, and when you do either .ceil() or .floor() you convert the VFR to CFR CORRECTLY. Without the fix, it would NOT convert to CFR correctly at all, nor would it be retained as VFR. (EDIT: When not fixed, the source will still be the right frame count, just the displayed FPS via clip properties would be completely wrong, it might as well say its 6969420.BlazeITUpL0L FPS in that state.)

[shph]

PRAGMA, If it helps to improve your function, here are 2 more very unusual short examples:
PHANTASMAGORIA_VTS_01_4-VC01 is possible to make it look more less ok if set input to progressive and VFM to 29.97 with field order set to 3 clip = core.vivtc.VFM(clip=clip, order=0, field=3), but after that in some places some moving objects sometimes show new interlaced artifacts. It also have a lot of additional rainbow/halo/ring artifacts, but this is not the subject of this discussion.
https://drive.google.com/file/d/1LDu...ew?usp=sharing

12OZMOUSE use some unknown pattern. it is 25fps PAL, but i gave up with that source because never was able to guess what is going on there. It looks like one type of interlaced layer was placed to other different type of interlaced layer during editing. As i know the source was DVD with Australian region.
https://drive.google.com/file/d/1K5W...ew?usp=sharing

[SaurusX]

Originally Posted by PRAGMA

Yes correct. I've noticed this in the intro as well. Again, seems to be caused by the dreaded Adobe bug.

Can you expand upon this? Are you saying Adobe Premiere has a known flaw that's known to cause this low res frame problem and maybe even the aliased red chroma problem? I've been trying to find an explanation for this for a long time.

[poisondeathray]

Originally Posted by SaurusX

Can you expand upon this? Are you saying Adobe Premiere has a known flaw that's known to cause this low res frame problem and maybe even the aliased red chroma problem? I've been trying to find an explanation for this for a long time.

It's not really an Adobe bug, per se. All NLE's will do something similar when the editor mishandles the source material. (You're supposed to remove pulldown before editing, and edit on a progressive timeline. But many editors are lazy, or don't know their job, or sometimes they are handed bad source material in the first place). The issue is most NLE's use a low quality deinterlace (something similar to a "bob" in avisynth, or just resizing a single field)

You can demonstrate the same degradation in vapoursynth or avisynth when you mishandle footage. It's essentially caused when you deinterlace a progressive frame, so only 1/2 the fields are used. 50% of the spatial information. (For the same reason, you don't want to QTGMC progressive frames, but QTGMC usually handles the issues more gracefully, usually, except when there are ghosting issues)

[PRAGMA]

Originally Posted by SaurusX

Can you expand upon this? Are you saying Adobe Premiere has a known flaw that's known to cause this low res frame problem and maybe even the aliased red chroma problem? I've been trying to find an explanation for this for a long time.

Correct. Adobe Premiere 6 for Mac (specifically "Premiere 6 for Mac", Not "Premiere Pro 6" or anything, Mac-specific).

It has a weird bug in relation to slicing and speed adjustment to spliced clips causing weird deinterlacing, reinterlacing, re-deinterlacing, multiple times depending on how it was sliced and adjusted. It could even go through it multiple times if it was mixed-scan (VST) and if output as interlaced, it would have interlaced progressive frames.

Even on the latest Adobe software it STILL doesn't handle mixed-scan (VST) content very well at all. It's sad.

EDIT: To be even clearer, this isn't just a case of the software just discarding half of a frame for a lot of frames or anything. This is the case of it re-interpolating said frames, then re-discarding over and over multiple times. If you ever notice a "shadow" bouncing up and down an outline/edge or invariant but patterned flickering/"sparks" then thank Adobe Premiere 6 for Mac (and possibly other earlier or later versions, but I know for SURE it happened on this version).