VoidWeave + VSGAN == Potential new Deinterlacing method

[PRAGMA]

In pvsfunc I recently added a Class method to PDeinterlacer called VoidWeave.
https://github.com/rlaPHOENiX/pvsfunc/blob/master/pvsfunc/pdeinterlacer.py#L152

It simply makes every other row on each field a 255 (#ff) RGB green. From there, it can be used with VSGAN and an inpainting model to fill the green with what the model believes should be their based on the context of the neighboring pixels.

This is a single-field deinterlacer like sangnom.

You can see an example use of this with VSMPEG
https://github.com/rlaPHOENiX/VSMPEG

Example of this being used with an in-the-works model:
https://streamable.com/2df9ig
*This is 25i->50p, Bob deinterlace*

As you can see, the Intro really struggles for whatever reason, but the actual live action scenes are really darn nice.

The problems I've encountered so far is that when using a bob tactic (every even (2nd) field) needs to be vertically aligned and to do that I opted for putting a black 1px border on the top, and remove 1px row of pixels from the bottom. It works, but it's definitely not as good as whatever way QTGMC is doing it.

As you saw in the intro, it is also clear that its doing a per-field deinterlace too, you can see clear as day vertical information being missing on some of the angled lines.

I should note, depending on how you train the model for VSGAN, can go reall quick. In the model im testing it gets 9.4 fps on my i7-8700k + 1080ti

Any advice?
:)

[ChaosKing]

Have you tried it on problematic sources? Could this also be used for ivtc for removing (undetected) combing?

Btw. I once tried MCBob() on a "it crowd" dvd and the result was perfect.

[PRAGMA]

Have you tried it on problematic sources? Could this also be used for ivtc for removing (undetected) combing?

Btw. I once tried MCBob() on a "it crowd" dvd and the result was perfect.

I wont lie, im not familiar with the whole "combing" problems, though I see talk about it all the time, and probably even encountered it myself.

Is this what you mean by combing?

https://thumbs2.imgbox.com/3b/a3/d6suxgOc_t.png (https://imgbox.com/d6suxgOc)

[ChaosKing]

hmm this looks like 2 problems combined

Look at the buttons here: https://forum.videohelp.com/threads/394868-How-to-deal-with-leftover-combing-after-ivtc#post2564552

Bu basically it's this something like this:
https://i.imgur.com/ruNUl62.png

It's also often found in the mouth region in animation bcs the change in the image itself is so small that the filter can't detect it properly.

[PRAGMA]

hmm this looks like 2 problems combined

Look at the buttons here: https://forum.videohelp.com/threads/394868-How-to-deal-with-leftover-combing-after-ivtc#post2564552

Bu basically it's this something like this:
https://i.imgur.com/ruNUl62.png

It's also often found in the mouth region in animation bcs the change in the image itself is so small that the filter can't detect it properly.

Ok I see the confusion.
So, with the image I showed, it seems it was caused by mvsfunc.ToRGB, something to do with the conversion caused the result to be very odd in edge cases (and only sometimes).

Regarding the image examples you sent on combing, this isn't going to be something related to this project (or even QTGMC unless you use Progressive mode).

With QTGMC or VoidWeave+VSGAN, it only runs on frames marked as Interlaced by the video metadata. Whereas stuff like TIVTC e.t.c tries to metrically detect a frame as interlaced, which is why you gotta fiddle with all kinds of detection settings unlike QTGMC or VoidWeave+VSGAN.

When using PSourcer and PDeinterlacer, it isn't really that difficult to manually mark a frame (that is burn-in interlacing, aka combed I guess) as interlaced, but I'm not sure how well it would actually work out.

[_Al_]

not knowing if it is related to your issues, recently realized, that interlace content has to be converted to rgb differently than as progressive, exactly because I noticed detail problems in combing areas after conversion, it needs to be treated:
#interlaced=True?
field_based = 0
try:
field_based = clip.get_frame(0).props['_FieldBased'] #0=progressive, 1=bottom field first, 2=top field first
except:
pass
if field_based:
#convert as interlaced
clip = clip.std.SeparateFields()
#here is conversion to rgb and then:
if field_based:
rgb = rgb.std.DoubleWeave()[::2]
this is for true interlace content and then_FieldBased would need to be always flagged, because it might be detected from source plugin or not.

With your content when actually fixing interlace artifacts, it might get more tricky to actually see what you have looking at rgb, perhaps you might never look at it, only after fixing :-)