Help with fixing "ghosting" luma that lasts for several subsequent frames

[Esn]

My apologies if this has been answered before - perhaps I'm not searching for the right terms. I've tried to solve this on my own, but haven't been able to...

I'm trying to rip an interlaced PAL DVD file, and have a pretty simple script:

QTGMC()
SelectEven()

But it seems that the original source was transferred to DVD in such a way that many scenes have a "ghosting" aftereffect like in the images below, with an afterimage (or "afterluma"?) of a frame remaining in the following frame, then gradually getting fainter over a few frames afterwards.

While a lot of scenes have this, other scenes (even ones with big color contrasts) don't seem to. I'm not sure why... maybe it's only the case for scenes with big luma contrasts?

When watching the original DVD, I can see the same thing somewhat as well, but somehow QTGMC() makes it a lot more obvious.

Does anyone have any suggestions for what I could try?

EDIT: a short video clip: files.fm/u/63h683ara

[Selur]

share a small clip of the unprocessed source not some screenshots,...

[Esn]

All right, here's a short video segment of the scene in the screenshots (saved losslessly in avidemux): https://files.fm/u/63h683ara
Is that okay?

[Esn]

share a small clip of the unprocessed source not some screenshots,...

Ok, I've added one to the 1st post, losslessly cut in avidemux (it's about a megabyte). Hope it's okay.

[Selur]

Okay, that sample isn't interlaced. Looking at it without deinterlacing I see no combing artifacts.
=> deinterlacing this will just make it worse
You need something heavy like MCTemporalDegrain + maybe something like DPIR.
Going to bed now, but just for the fun of it: Here's what happens if you would use VapourSynth with BasicVSR++ followed by MCTemporalDegrain: https://files.fm/u/wu7st2j2r

Cu Selur

[Esn]

Thank you. I love that the ghosting is gone, though I wonder if the textures are a bit too smooth now.
I'll try to experiment. What is "DPIR", though? I couldn't find it. I also can't find "MCTemporalDegrain"... I can find "TemporalDegrain" and "MCTemporalDenoise" - is it one of those?

Okay, that sample isn't interlaced
When I do the preview of the original VOB in avisynth, it definitely looks interlaced. Attaching a screenshot of the same scene with no filters.
I created a .d2v file with DGIndex (by making a selection and then pressing F4. I had "Honor pulldown flags" marked under Video --> Field operation) and loaded it like this:

MPEG2Source("VTS_03_2.d2v", cpu=0)

But when I go through the original VOB frame-by-frame in SMPlayer, or VLC Player, or avidemux, I see no interlacing. I think there must be something really basic that I'm missing here...

May I please know how you're loading that MKV file that I posted into avisynth? (the exact line of code, I mean... I'm still kinda new to this)

EDIT: External link to image because I'm not sure if my attachment can be viewed: https://imgur.com/a/2klfW1e

EDIT2: I think this might be relevant... when making the .d2v file, I get the following message: "A field order Transition was detected. It is not possible to decide automatically if this should be corrected. Refer to the DGIndex users manual for an explanation. You can choose to correct it by hitting the Yes button below or you can correct it later using the Fix D2V tool. Correct the field order Transition?"

If I select "Yes", then the original .d2v gets saved with a .d2v.bad extension, and a "fixed" one also gets saved. When I opened the "fixed" one in avisynth, it looks interlaced (with combing) like the screenshot above. But when I open the "bad" one instead (first changing the extension from .d2v.bad to .d2v), it now no longer looks interlaced (though you can still see the "ghosting" that I mentioned in the first post):
https://imgur.com/a/LFCzKIe
Should I just use the "bad" d2v file and ignore the "fixed" one?

[hello_hello]

PAL DVDs sometimes have their fields "out of phase" for some reason. So instead of having the top and bottom fields of a progressive frame together, something like this:

A B C D E
A B C D E

it's something like this

A B C D E
B C D E F

and because the fields are from different frames they're different moments in time, so the video looks interlaced (combed) when it's not.

One way to check is to put TFM at the beginning of a script with it's de-interlacing disabled. If the video is progressive and TFM can match the fields up and the "combing" goes away, that's all you need. If there's still "combing" it's probably really interlaced and needs de-interlacing.

I don't know what the deal is with your d2v file, but I remuxed your mkv sample as an m2ts file with TSMuxer so I could open it with DGIndex and there was no combing or TFM required. Whichever d2v file you use though, adding TFM(pp=1) to the script will hopefully ensure the output is the same anyway.

Your sample could have been a couple of nano-seconds longer, but there's obviously "ghosting". It's part of the video though so I don't know if there's a way to remove it without turning everything else to mush.

[Selur]

DPIR for Avisynth: https://github.com/Asd-g/AviSynth-DPID
yes, 'MCTemporalDenoise' was what I meant. :)

Cu Selur

[anton_foy]

DPIR for Avisynth: https://github.com/Asd-g/AviSynth-DPID
yes, 'MCTemporalDenoise' was what I meant. :)

Cu Selur

I think you mean this: https://github.com/Asd-g/avs-mlrt

DPID is not at all the same as DPIR :)

[Esn]

@hello_hello,
thanks for the explanation. It doesn't seem that the fields ARE "out of phase" in this one, though. I had assumed they were because I trusted DGIndex's "fixed" d2v. But the supposedly "bad" .d2v file seems to look just like it should, while the "fixed" one adds combing. I've found some others (http://macilatthefront.blogspot.com/2019/01/tutorial-how-to-convert-dvd-for-editing.html) online mentioning the same thing, maybe the common element is if the source is animation that has duplicate frames sometimes...

TFM() or TFM(pp=1) seems to make no difference at all to the output.

@Selur,
I wasn't sure what to do with MCTemporalDenoise, so I just tried the MCTemporalDenoise examples (http://avisynth.nl/index.php/MCTemporalDenoise#Examples) given on the wiki, and noticed no difference to the "ghosting" effect. Actually, from the examples given in the second post on the forum thread for MCTemporalDenoise (https://forum.doom9.org/showthread.php?t=139766), it seems that what it mainly is for is to make textures more smooth? But that's not what I'm after... I mainly want to remove the "ghosting", the afterimages of previous frames (or perhaps it's just the afterimages of the brightness/luma of the previous frame, rather than the image itself). Can MCTemporalDenoise do that?

Also, may I please know what settings you used for BasicVSR++ & MCTemporalDenoise in VapourSynth with those examples from yesterday?

@anton_foy,
Thanks for the link. Wow, that's a lot to study (https://github.com/AmusementClub/vs-mlrt/wiki). Do you have any suggestions for what to try in there?
I notice that a lot of people seem to be recommending VapourSynth plugins or ports of them. Is VapourSynth preferred over avisynth these days?

[Selur]

Argh, correct. :)

[kedautinh12]

MCTD is latest ver of MCTemporalDenoise
https://github.com/Asd-g/AviSynthPlus-Scripts/blob/master/MCTD_.avsi

[Esn]

MCTD is latest ver of MCTemporalDenoise
https://github.com/Asd-g/AviSynthPlus-Scripts/blob/master/MCTD_.avsi
Thank you! Do you have any recommendations for what I should try with the video clip? (files.fm/u/63h683ara)

I'm not actually even sure what the right name for the problem I'm seeing is! I would say "ghosting", except that "ghosting" is described as "a faint copy of the picture offset horizontally", and that's not what this is at all! (rather, it's the faint copy of the previous frame, particularly of its luma, on subsequent frames)

[anton_foy]

Thanks for the link. Wow, that's a lot to study. Do you have any suggestions for what to try in there?
I notice that a lot of people seem to be recommending VapourSynth plugins or ports of them. Is VapourSynth preferred over avisynth these days?

I would like to help you but sadly mlrt with dpir option is too heavy for my old and slow gpu. Managed to get it to work if I downscaled by half but then I cannot tell really if it is good or not. My suggestion for denoising is something like smdegrain(tr=6,mode="temporalsoften") followed by a spatial cleaning such as neo_vd() that is neo vaguedenoiser.

[kedautinh12]

Maybe, some extender model from here can help you
https://upscale.wiki/wiki/Model_Database

[Esn]

Your sample could have been a couple of nano-seconds longer
Here's a slightly longer one with more scenic variety. I saved it as a .ts file this time; I hope that'll make it easier to import into avisynth (?):
files.fm/u/e2v4kprk7
(P.S. the issue with the sound is a separate thing - I fixed that on my own already. Whoever made this DVD really screwed up in more ways than one...)

You can also see that there often ISN'T a "ghost" in subsequent frames even when the colour changes radically in one part of the image (like when the red banners pass quickly in front of the horses) - rather, it's big changes in luma in particular that seem to cause the so-called "ghosting" (though again, I suspect this may not be the right term).

[Selur]

For the combing artifacts in the later half of the clip 'TFM()' seems to work fine.
Here's (https://files.fm/u/v9b6rqtfh) what you get if TFM()+vs-grlir (https://github.com/HolyWu/vs-grlir) is used. (vs-grlir is only available for Vapoursynth an even with modern hardware slow)