vs-mfdin

[poisondeathray]

Thanks HolyWu

https://github.com/HolyWu/vs-mfdin

[Selur]

Sounds interesting, has anyone compared this to QTGMC?

[poisondeathray]

Sounds interesting, has anyone compared this to QTGMC?

Just playing with it now, it's quite slow - a few times slower than QTGMC

So far results are smooth temporally, but loss of high frequency details like a denoiser was applied

There were at least 2 models in the MSU benchmark - MFDIN-L is the one that scored the highest. It's unclear which model this one is

[Emulgator]

MFDIN-L is also where I put my hopes in for the future of deinterlacing,
QTGMCp being my preferred choice so far and staying in the toolkit.

[poisondeathray]

I think "MFDIN_old_2P.pth" was for the real old videos - the low quality test videos with compression artifacts. I don't think it was MFDIN-L .

The benchmarks like PSNR, SSIM are easy to "trick", so I still would like to see and test real models , real videos, preferably not in the testing or training sets

[Selur]

So far results are smooth temporally, but loss of high frequency details like a denoiser was applied
Maybe using QTGMC and feeding a with vs-mfdin created deinterlaced clip as EdiExt can help with the detail preservation.
(this is a way one of the ways hybrid does support https://github.com/pifroggi/vs_deepdeinterlace atm. )

[poisondeathray]

Maybe using QTGMC and feeding a with vs-mfdin created deinterlaced clip as EdiExt can help with the detail preservation.
(this is a way one of the ways hybrid does support https://github.com/pifroggi/vs_deepdeinterlace atm. )

Possibly - it would take a long time and many tests to see if it's worthwhile

Are you or hybrid users seeing any benefit using vs_deepdeinterlace in that way ? Any comparions posted ?

MFDIN-L gets about 4dB higher than the base model; even if PSNR is a crappy test, 4dB is significant.

[Selur]

I did some testing back when I added vs_deepdeinterlace, I did spot some differences on some samples but didn't do any in-depth comparison.

[Emulgator]

I was only looking at "Tokyo Market" sample and decided that my MFDIN-L candidates
would come from best-effort analog captures like Beta, VHS, Video8, not maimed further.

To cope with additional compression on top:
I did not dare to hope if that can be achieved in one go, good news if indeed possible.
And maybe mending shimmer, chroma field bleeding, and progressive resizing, and...
all that mess that misunderstandings/shortcuts/assumptions/bitrate starvings introduced with every generation.

[poisondeathray]

In the demo 2 years ago, the examples posted were degraded, sometimes with severe artifacts. The GT versions provided were fairly clean. My impression was that version of MFDIN posted was intended for problematic interlaced video . That dataset link is still active

Anyways I posted on github and asked for clarification of the version of the provided model , and if the other model would be posted

[poisondeathray]

Anyways I posted on github and asked for clarification of the version of the provided model , and if the other model would be posted


“MFDIN_old_2P.pth” represents the version of the model enhanced for older films mentioned in the paper.
MFDIN-L, on the other hand, is the Large version, which is not planned for release at this time.


:(:(:(

[Selur]

Does https://github.com/HolyWu/vs-mfdin only work on TFF content? (or am I doing something wrong?)
(tried with this (https://drive.google.com/file/d/1AhqpN0uMOYl9iu1tiWSmPmoUuNCrhnoA/view?usp=drive_link))
I tried using:

clip = clip.std.SeparateFields()
clipodd = clip[1::2] # Odd (top) fields
clipeven = clip[::2] # Even (bottom) fields
clip = core.std.Interleave([clipodd, clipeven])
clip = clip.std.DoubleWeave(tff=True).std.SelectEvery(2,0)

(which should change bff to tff, right?)
before applying vs-mfdin, but the output still is jumpy as if the field order was wrong during bobbing.
(script I used: https://pastebin.com/3xVwvh2P)

Cu Selur

[cubicibo]

A FlipVertical on a BFF clip makes it TFF.


vclip = core.std.FlipVertical(clip) # clip is BFF
vclip = core.std.SetFieldBased(vclip, 2) # ensure vclip is tagged as TFF
deint = ... # apply mfdin on vclip
output = core.std.FlipVertical(deint) # cancel flip

[_Al_]

clip = core.std.Interleave([clipeven, clipodd]) # <-swapping them to have even first


It looks ok, just when interleaving, should not be those fields swapped?

[Selur]

@_AI_: I thought a swapped the fields, by moving the clipeven to the front. (0= even, 1 = odd, so Interleave([clipeven, clipodd]), should change the order)
To be sure, I tried with:
# deinterlace using MFDIN

# convert to TFF, since MFDIN only supports TFF
clip = clip.std.SeparateFields(tff=False)
clipodd = clip[1::2] # Odd (top) fields
clipeven = clip[::2] # Even (bottom) fields
clip = core.std.Interleave([clipeven, clipodd])
clip = clip.std.DoubleWeave(tff=True).std.SelectEvery(2,0)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_TOP) # tff

from vsmfdin import mfdin
clip = mfdin(clip=clip, device_index=0, double_rate=True, trt=True, trt_cache_dir=r"J:\TRT")
but the output is still jumpy.

@cubicibo: I like the idea. :)
I tried:
# convert to TFF, since MFDIN only supports TFF
vclip = core.std.FlipVertical(clip)
vclip = core.std.SetFrameProps(clip=vclip, _FieldBased=vs.FIELD_TOP)
from vsmfdin import mfdin
clip = mfdin(clip=vclip, device_index=0, double_rate=True, trt=True, trt_cache_dir=r"J:\TRT")
# undo the tff vertical flip
clip = core.std.FlipVertical(clip)
and it worked! Thanks!

Cu Selur

Ps.: Added it to the latest Hybrid dev as a stand alone deinterlacer and in QTGMC as EdiExt value.