Idea for unusual deinterlacing method for retro games

[Eyedunno]

Hello, this is my first time posting here, and my request (if it doesn't yet exist) is a little unusual, so please bear with me. I already posted it on VirtualDub Forums, and they thought somebody here might be more knowledgeable about whether this exists and/or how to make a relevant Vdub or avisynth filter. So without further ado...

I like to capture old video games with a capture card, but there are serious issues with deinterlacing things that have a 60fps flicker. Using half fields will either show a static image of the thing that is flickering or show nothing at all. Using blended fields will show static, semi-transparent objects.

A blended clip will generally blend the top and bottom fields with equal, 50%/50% opacities. But I wondered, "what if you were to blend a frame with a 75% top field/25% bottom field ratio, and the next frame with a 25% top/75% bottom ratio, and so on?"

I did a little proof of concept, but first a little warning: I don't have a website and didn't really feel like setting one up just for this, so I made a single HTML file with embedded base64 images, so it may take a minute or so to load, even on a fast connection. They are animated PNGs, so they will not show up properly in browsers other than Firefox or Opera. If you are using MSIE, Chrome, or Safari, sorry, nothing to see here. With that out of the way, here is the proof of concept:

http://www.dinkypage.com/161836

The important images are the first and last ones - the first one is a 60fps animation of what the flickering should look like, while the last one is my 30 fps proof of concept. Everything in between is the (inferior) 30fps alternatives. It seems to me that not only does the 75/25 method give the illusion of the same kind of flicker, but, although I haven't tested it further, it would most likely look pretty good with things in motion too - kind of like a compromise between top-or-bottom-fields-only and blended fields.

Here's an example of the kind of interface that might work:

top                 FIELD OPACITY ON                bottom
field              ODD-NUMBERED FRAMES              field
|--------------|----------------------------------------|
      25%                         75%

Even-numbered frames would have the opposite opacities (so 75% top field/25% bottom field for the example above). 50%/50% would be the same as simply blending the fields equally. 0%/100% and 100%/0% would be fairly silly settings where top and bottom fields would alternate from one frame to the next. Anything in between would be kind of a compromise between the sharpness of discarding one field and the relative fluidity of motion of a blended clip, with 25%/75% and 75%/25% being probably the best for dealing with huge changes between the two fields (such as 60fps flickering).

So is there a filter that deinterlaces with an algorithm like this? If not, does anybody have the chops to make one? It would seem to be a rather simple thing, and it would also seem to be a good middle ground between the two simplest deinterlacing methods that doesn't require much more processing power than regular blended fields.

Thanks.

[jackoneill]

If you don't mind 60 frames per second output:

qtgmc(preset="placebo")

(No, it doesn't do the 25/75 thing you describe. Read about it here: http://forum.doom9.org/showthread.php?t=156028.)

[Gavino]

I haven't investigated if what you are trying to do is a good idea, but this script implements your scheme directly:

WhateverSource(...) # appropriate source filter
Bob(height=height)
b = SelectEven() # bottom fields interpolated to full height
t = SelectOdd() # ditto for top
eb = b.SelectEven() # from even frames
et = t.SelectEven()
ob = b.SelectOdd() # from odd frames
ot = t.SelectOdd()
Interleave(Merge(eb, et, 0.75), Merge(ob, ot, 0.25))

[Eyedunno]

Originally Posted by Gavino

I haven't investigated if what you are trying to do is a good idea, but this script implements your scheme directly:

WhateverSource(...) # appropriate source filter
Bob(height=height)
b = SelectEven() # bottom fields interpolated to full height
t = SelectOdd() # ditto for top
eb = b.SelectEven() # from even frames
et = t.SelectEven()
ob = b.SelectOdd() # from odd frames
ot = t.SelectOdd()
Interleave(Merge(eb, et, 0.75), Merge(ob, ot, 0.25))

Wow, that looks amazing. It's been years since I've done anything with AVIsynth, but I will give it a try when I get home, both with Final Fantasy V when targeting all enemies, and with a more standard, action-oriented game.

[cretindesalpes]

This thread might be interesting too:

60 -> 30 fps flickering with minimum blur

[Eyedunno]

Aha! Good to know this idea isn't original to me then! (Shows how bad my search terms were!) And maybe 67/33 is a better ratio. I will definitely have to try both in a few hours when I'm home.

[mastrboy]

Dealing with such content you would probably get better help over at TASVideos since they specialize in that exact kind of content: http://tasvideos.org/EncodingGuide.html and http://tasvideos.org/forum/viewforum.php?f=52

And this huge script created for dealing with games: http://tasvideos.org/EncodingGuide/HybridEncodeScript.html

[Eyedunno]

Well, they don't specialize in that EXACT kind of content. This is capture card content, so interwoven 30fps frames rather than proper 60fps frames. Not a huge difference though.

Anyway, I tweaked Gavino's script a little. SeparateFields() provides cleaner results with my source material than Bob(height=height) (any resizing can be done in VirtualDub). Also, I decided to use 0.666 and .334 instead of .75 and .25.

It works fantastically for what I hoped it would work for - 60 Hz flicker and general onscreen movement. Unfortunately, it's not so hot for 30 Hz flicker. It's not illustrated in my 2-frame example, but Final Fantasy V actually has two simultaneous flicker effects on this screen - a 60 Hz flicker on the "target all" hand icon, and a 30 Hz flicker on the character sprites that are affected by status effects. If the timing isn't right, this kills the 30 Hz flicker.

I came up with an alternative, but it's hardly ideal. Instead of two values, I went with four values to preserve the 60 Hz flicker and kind of show a hint of the 30 Hz flicker. Hardly ideal though; I will probably end up using TASBlend as written, with an additional SeparateFields() line to make my actual console output similar to emulator output.

function AltBlend(clip c)  { 
        c 
        Interleave(Layer(SelectEvery(8, 0), SelectEvery(8, 1), level=52), 
        \          Layer(SelectEvery(8, 2), SelectEvery(8, 3), level=156), 
        \          Layer(SelectEvery(8, 4), SelectEvery(8, 5), level=104), 
        \          Layer(SelectEvery(8, 6), SelectEvery(8, 7), level=208)) 
}

AviSource("...")
SeparateFields()
AltBlend()

Anyway, thanks, everyone. I've learned (or relearned :P) a few things about AVISynth in the process of screwing with this. Here is some YouTube video showing TASBlend and my slight modification at work.
http://www.youtube.com/watch?v=VxZ80dl2AiY

[pandy]

AFAIK how video is generated then many video sources provide signal which is not interlaced and this signal can't be captured by capture device expecting interlace signal - in other words some issues are introduced during capture and it can be difficult to fix them at the postprocessing stage.
Perhaps best method to deal with such signal is to shift top field down by half of the line and bottom shift half line up then create 50/60p stream with vertical resolution half of normal TV resolution ie 288/240p.

However best should be use capture device in non interlace mode.

[Eyedunno]

Originally Posted by pandy

AFAIK how video is generated then many video sources provide signal which is not interlaced and this signal can't be captured by capture device expecting interlace signal - in other words some issues are introduced during capture and it can be difficult to fix them at the postprocessing stage.
Perhaps best method to deal with such signal is to shift top field down by half of the line and bottom shift half line up then create 50/60p stream with vertical resolution half of normal TV resolution ie 288/240p.

However best should be use capture device in non interlace mode.

The signal is interlaced, and the capture card picks it up fine. And actually, shifting the fields (as the Bob filter does) created a problem where it literally bobbed. Simply splitting the fields results in a nice, stable picture. BTW, I've since used the TASBlend filter for another, much longer video (longer than 10 minutes), and it worked great. This is going to be my default gaming deinterlace method.

[pandy]

Originally Posted by Eyedunno

The signal is interlaced, and the capture card picks it up fine. And actually, shifting the fields (as the Bob filter does) created a problem where it literally bobbed. Simply splitting the fields results in a nice, stable picture. BTW, I've since used the TASBlend filter for another, much longer video (longer than 10 minutes), and it worked great. This is going to be my default gaming deinterlace method.

How many lines is in video, what is video frequency - i mean is this is 525?, 524? or 526? lines video (i assume it is NTSC-like video source)

[Eyedunno]

Originally Posted by pandy

How many lines is in video, what is video frequency - i mean is this is 525?, 524? or 526? lines video (i assume it is NTSC-like video source)

Hah. The SNES, on almost all games, uses 224 line video (either 256X224 or 512X224). This naturally gives most modern TVs a heart attack.

There is a 512X448 video mode, but it is limited to 16 onscreen colors at once, and this made it quite unpopular to developers. One game that used it extensively was RPM Racing.
[url]http://www.youtube.com/watch?v=OyvecUB5xp4[\url]

[pandy]

my point is that typical video generators used in games produce signal which is almost compatible with TV - difference is very small - one line - counter cant count to half - so most of the equipment produce or 524 or 526 lines video and this video is not interlaced - thanks to 525 lines divided by 2 (two fields) there is half lineshift which create interlacing (lines partially overlapping) - for those quasitv sources there is no half lines - sometimes there is short and long line (each frame it is switched to produce quasi 525 line video by mixing 524,526,524,526 etc). Forcing interlace capture device to capture progressive video give strange - quasi-interlace effect.
Try to use DScaler (AFAIR it provide special mode for 524,526line videos) to capture such sources and then compare results - perhaps deinterlacing will be not required.

[gyth]

The problem isn't with deinterlacing, you have a solid, properly-flickering 60fps signal after the SeparateFields.
The problem is with framerate reduction, so you can post on YouTube, which caps out at 30fps.
Your solution is serviceable, but there are improvements that can be made.

For every 4 frames you only get to keep 2.
SelectEvery(4, 0,2) - Picking every other one is in some senses ideal, but flickering is lost.
SelectEvery(4, 0,1) - Picking the frames asymmetrically will preserve the flicker, but motion becomes uneven.
Interleave(Layer(0,1), Layer(2,3)) - Layering everything preserves most effects, but motion becomes blurred.

The trick is, it isn't all or nothing.
If the parts of the frame that are flickering can be identified then the "reflickering" can be applied to just that.
That is the function I'm working on over at TASVideos.org.
If you'd like to hear more you can ask here, or visit over there.

[pandy]

Originally Posted by gyth

The problem isn't with deinterlacing, you have a solid, properly-flickering 60fps signal after the SeparateFields.

Problem is that video is generated 60 times per second without interlacing and it is captured with capture device designed for interlace video.

[Eyedunno]

Originally Posted by pandy

Try to use DScaler (AFAIR it provide special mode for 524,526line videos) to capture such sources and then compare results - perhaps deinterlacing will be not required.

Hrm, I do use DScaler (Win7 doesn't have drivers for TV@nywhere as far as I know, and I prefer DScaler anyway), but I have not noticed such an option. The only options are for pixel WIDTH (720, 640, etc.). I think this varies by card though, as my older BT878 card had a different set of options.

Edit: Oh, also, as far as capturing, there are very few deinterlacing options. I can either capture odd fields, even fields, blended fields, or full, interwoven frames. The latter option creates rather huge files using the Lagarith codec, but I can get any kind of deinterlacing I want in post-processing.

Originally Posted by gyth

The trick is, it isn't all or nothing.
If the parts of the frame that are flickering can be identified then the "reflickering" can be applied to just that.
That is the function I'm working on over at TASVideos.org.
If you'd like to hear more you can ask here, or visit over there.

It sounds fantastic, like if you could detect a 30Hz flicker and ensure that it alternates frames correctly or detects 60Hz flicker in another part of the image and implements a scheme like TASBlend over just that area. I do think that even if you get this working well, it will probably be more useful for TASVideos than for me. I try to capture in S-Video when possible, but even if I were capturing in RGB, it wouldn't be as clean as an emulator signal (though it would be close, especially if I had a 1chip SNES).

Still, I am very interested in what you come up with.