Speeding up scanlines script

[creaothceann]

I want to encode a speedrun of the DOS game Epic Pinball (http://tasvideos.org/2830S.html) (emulated on JPC-RR (http://tasvideos.org/EmulatorResources/JPC.html)), but with scanlines that extend the height from the 640x400 source to a 4:3 ratio.

(The source file I'm using is available at the first link. It's in YV12, but in the future my own recordings will be in RGB.)

Currently I add the scanlines like this:

DSS2("dwangoAC-WIP-Epic-Pinball-05.mkv") # YV12 640 x 400
ConvertToYUY2 # Crop can't do odd numbers in YV12
PointResize(Width, Height * 5).Scanlines # add scanline after every 5 lines
BilinearResize(1280, 960) # resize to final video size


function Scanlines(clip) {
clip
top = Crop(0, 0, 0, 5).AddBorders(0, 0, 0, 1)
bottom = Crop(0, 5, 0, 0)
bottom = (bottom.Height > 5) ? bottom.Scanlines : bottom.AddBorders(0, 0, 0, 1)
StackVertical(top, bottom)
}
Images:
source (http://img696.imageshack.us/img696/6919/0400.png)
with scanlines (http://img52.imageshack.us/img52/1803/2400.png)
final size (http://img824.imageshack.us/img824/8869/1024.png)

Is there a way to speed up this script a bit?

[Didée]

o = last.converttoyv12() # most masktools filters work only on YV12
lines = last.mt_lutspa(relative=false,expr="y 5 % 0.01 < 55 y 5 % 1.1 < 200 y 5 % 3.9 > 200 255 ? ? ?")
last.mt_lutxy(lines,"x y * 255 /",U=2,V=2)
#interleave(o,last)
return(last)

This renders a 640x480 clip at ~600 (six hundred) fps on my i860. Fast enough? :)

[Robert Martens]

I can't hit nearly the speed Didée's technique does (~300 fps on my Q6600 for his, versus ~75 for my approach), but you can achieve results like your original sample frames without masktools, if for some reason you don't want to use the plugin (not that I'd advocate ignoring it, mind you):

game = DSS2("dwangoAC-WIP-Epic-Pinball-05.mkv").ConvertToYUY2().BilinearResize(640,480)
scanlines = Blackness(game)

Interleave(game,scanlines)
AssumeFieldBased()
Weave()
AssumeFrameBased()
BilinearResize(1280,960)


I start out by loading the clip and resizing it to 480 height; the resulting proportions are the same as padding with scanlines.

Then it's a simple matter of generating a black clip with the same frame size, rate, colorspace, etc. as your source material. Once you interleave and weave them, you have an interlaced clip, but AssumeFrameBased takes care of that and all that's left is to scale up to your final dimensions.

I don't suppose this will compress very efficiently, though; out of curiosity, may I ask why you want to add scanlines instead of just upscaling the footage?

[creaothceann]

o = last.converttoyv12() # most masktools filters work only on YV12
lines = last.mt_lutspa(relative=false,expr="y 5 % 0.01 < 55 y 5 % 1.1 < 200 y 5 % 3.9 > 200 255 ? ? ?")
last.mt_lutxy(lines,"x y * 255 /",U=2,V=2)
#interleave(o,last)
return(last)

This renders a 640x480 clip at ~600 (six hundred) fps on my i860. Fast enough? :)
Yeah, but... it looks a bit different to what I'd expect, when I think about it. All I'd need to do is PointResize to height * 6 (5 source lines and 1 line that will be blanked out), apply a mask that does the blanking, and scale back to the final resolution. Or something with a mask whose height is only that of the final height.
I can't really see what that expression does... :scared: is there a tutorial?

EDIT: I think it does this:
if modulo(y, 5) < 0.01 then 55 else
if modulo(y, 5) < 1.1 then 200 else
if modulo(y, 5) > 3.9 then 200 else 255


I start out by loading the clip and resizing it to 480 height; the resulting proportions are the same as padding with scanlines.
[...] out of curiosity, may I ask why you want to add scanlines instead of just upscaling the footage?
I want to keep the look of a CRT screen (http://img97.imageshack.us/img97/8387/doswv.png) - a simple bilinear upscale doesn't achieve that. The scanlines add only 20% to the height of the image, so I can't just interleave it with black lines.

Ideally there would be a device with a resolution of 3200x2400 pixels; barring that I have to use blending, with the final number of lines being as high as possible.

[Didée]

All I'd need to do is PointResize to height * 6 (5 source lines and 1 line that will be blanked out), apply a mask that does the blanking, and scale back to the final resolution.
Do you insist on resizing to height*6, or may we omit that part? (You wanted to have it fast, nay?)


Same method, without any interpolation. Original pixels are preserved, plus 1 black line after each 5 original pixels.
o = last.pointresize(width,480).converttoyv12() # most masktools filters work only on YV12
lines = o.mt_lutspa(relative=false,expr="y 6 % 0.01 < 0 255 ?")
last.mt_lutxy(lines,"x y * 255 /",U=2,V=2)
#interleave(o,last)
return(last)

Still, 600 fps.

* * *

Edit: Wait, no ... that's still not exactly what you want.

Okay, for the moment, and for the sake of simplicity, here it is with 6*up- & downsizing. Hopefully this is what you're aiming for?

o = last.pointresize(width,400*6).converttoyv12() # most masktools filters work only on YV12
lines = o.mt_lutspa(relative=false,expr="y 6 % 0.01 < 0 255 ?")
last.mt_lutxy(lines,"x y * 255 /",U=2,V=2)
spline64resize(width,960).spline36resize(1280,960)
return(last)

Speed isn't earthshattering anymore .... ~50 fps (singlethreaded), or 150~160 fps (multithreaded: SetMTmode(2,6) ). But that should be acceptable ... with your initial script I get 3.75 fps (singlethreaded), so that's a 13-fold speedup. ;)

[Usedocne]

I always liked me some scanlines when playing emulators or watching anime, nice to see such a niche get a bit of attention.

Oh yeah, Didée how would I tweak your script for 75% transparency?

thx

[creaothceann]

Do you insist on resizing to height*6, or may we omit that part? (You wanted to have it fast, nay?)
I don't see how, since there has to be an increase in height somewhere. This seems to preserve the lines the most.

The only alternative imo would be to create a "target frame" with the final height, fill it using the source data similar to BilinearResize, and apply a mask on it which also has the final height. I just don't know how this could be done.

Okay, for the moment, and for the sake of simplicity, here it is with 6*up- & downsizing. Hopefully this is what you're aiming for?
[...]
Speed isn't earthshattering anymore .... ~50 fps (singlethreaded), or 150~160 fps (multithreaded: SetMTmode(2,6) ). But that should be acceptable ... with your initial script I get 3.75 fps (singlethreaded), so that's a 13-fold speedup. ;)
Yes, that's what I've been coming up with as well before your edit, using your code. :)
This is my current version:

DSS2("dwangoAC-WIP-Epic-Pinball-05.mkv")
Add_PC_Scanlines(960) # function is only for 640x400 sources


function Add_PC_Scanlines(clip src, int LineCount, bool "spline") {
src
PointResize(Width, Height * 6)
Mask = mt_lutspa(false, "y 6 % 0.01 < 0 255 ?")
mt_lutxy(Mask, "x y * 255 /", U=2, V=2)
y = LineCount
x = y * 4 / 3
return default(spline, false) ? BilinearResize(x, y) : Spline64Resize(Width, y).Spline36Resize(x, y)
}
Instead of 600 MB RAM it takes much less now, though there may be a memory leak? RAM usage seems to be climbing when I watch AvsP or VDM in the task manager.

Maybe I have to use "last" more in my code, like you do, though it seems not necessary.


I always liked me some scanlines when playing emulators or watching anime, nice to see such a niche get a bit of attention.
I have to add that emulators for TV-based consoles (e.g. SNES) have to use a different method. On the SNES the lines are stretched horizontally to a 4:3 format, and the scanlines between them are a bit different too.

Here's a nice thread: http://board.byuu.org/viewtopic.php?f=10&t=147

[Didée]

Instead of 600 MB RAM it takes much less now, though there may be a memory leak? RAM usage seems to be climbing when I watch AvsP or VDM in the task manager.
That's Avisynth filling up its frame cache while the script is running. The amount of framecache can be controlled with SetMemoryMax().
I tried, and with SetMemoryMax(256) the Ram usage stops at ~350'000 kB, which is reached after roughly ~800 frames.

On another note, with the odd relation of 8/10 it seems hardly possible to get a "clean" scanline effect ... the spacing is not distributed evenly. Make a deep zoom, you see what I mean.

[creaothceann]

That's Avisynth filling up its frame cache while the script is running. The amount of framecache can be controlled with SetMemoryMax().
I tried, and with SetMemoryMax(256) the Ram usage stops at ~350'000 kB, which is reached after roughly ~800 frames.
I see.
Well, with that my initial question is answered, I think. Thank you! :)

On another note, with the odd relation of 8/10 it seems hardly possible to get a "clean" scanline effect ... the spacing is not distributed evenly. Make a deep zoom, you see what I mean.
Mmmh, it should still look somewhat clean when a line count is chosen that is a divisor of 2400. I used 960 because my TFT's native resolution is 1280x1024.
I guess that's the best image quality possible though.

[ajp_anton]

o=imagesource("0400.png").converttoyuy2 #Loaded your source image
o
interleave(blankclip(last,framecount),last) #Switch places if clip is RGB
assumefieldbased.weave #Every 2nd line is black.
pointresize(width,int(height*1.5)) #Every 3rd line is black.
interleave(o.pointresize(width,height),last)
assumefieldbased.weave #Every 6th line is black.
bilinearresize(1280,960)
300fps until the final resize, 70fps with it. Singlethreaded, i7 @ 3.3GHz.

edit:
imagesource("0400.png").converttoyv12
o=pointresize(width,height*6)
blankclip(o,2).mt_lutspa(relative=false,expr="y 6 % 0 == 255 0 ?")
mt_merge(o,blankclip(o),pointresize(width,height))
bilinearresize(1280,960)
200fps before resize, 70fps with. Singlethreaded, i7 @ 3.3GHz.
Here you can easily change all "6"s (there's only two) into how often you want the black line to appear. And with "% 0" you can control where the black lines will be.

[creaothceann]

Thanks! That's an interesting idea.

I had to change it to "% 5" to make the last line in each 6-line group black. This results in a slight difference when I use this code:

DSS2("dwangoAC-WIP-Epic-Pinball-05.mkv")
v1 = Add_PC_Scanlines1(720)
v2 = Add_PC_Scanlines2(720)

v1 # change to see difference
#Crop(0, 0, Width, 500)


function Add_PC_Scanlines1(clip src, int LineCount, bool "spline") {
src
PointResize(Width, Height * 6)
Mask = mt_lutspa(false, "y 6 % 0.01 < 0 255 ?")
mt_lutxy(Mask, "x y * 255 /", U=2, V=2)
y = LineCount
x = y * 4 / 3
return default(spline, false) ? BilinearResize(x, y) : Spline64Resize(Width, y).Spline36Resize(x, y)
}



function Add_PC_Scanlines2(clip src, int LineCount, bool "spline") {
src
#converttoyv12
o=pointresize(width,height*6)
blankclip(o,2).mt_lutspa(relative=false,expr="y 6 % 5 == 255 0 ?")
mt_merge(o,blankclip(o),pointresize(width,height))
y = LineCount
x = y * 4 / 3
return default(spline, false) ? BilinearResize(x, y) : Spline64Resize(Width, y).Spline36Resize(x, y)
}

I guess at this point this difference doesn't matter much for my purposes though, and I could also leave it at "% 0".

[Dogway]

very interesting theme. Hope it develops beyond the scanline thing and introduces some other CRT phosphor based aspects.

He posted the MATLAB code for his emulator filter here (http://board.byuu.org/viewtopic.php?f=10&t=147&start=270)

[2Bdecided]

CRT simulation - brilliant!


You've got to be careful adding black scan lines. (1) It reduces the overall brightness, and (2) with yv12 it can visibly trash the chroma.

Solution to (2) is easy: only process the luma - leave the chroma as-is. Or use YUY2. Depends what your target is.

Solution to (1): No solution! Possible improvements: either don't make the black lines totally black, and/or increase the overall brightness to compensate. Problem with the latter is that you'll clip highlights.


I haven't done what you're doing - I've tried preserving the interlacing as interlacing on a progressive display by sending individual fields with the "other" field represented by black or at least darker lines. It works, but you need frame locked video replay, which is impossible at 50Hz on most TFTs (some do it). It's horrible to try to encode something like this though - it requires an insanely high lossy bitrate to preserve the effect cleanly. I think you'd face the same problem (though not as bad).

Cheers,
David.

[creaothceann]

You've got to be careful adding black scan lines. (1) It reduces the overall brightness, and (2) with yv12 it can visibly trash the chroma.

Solution to (2) is easy: only process the luma - leave the chroma as-is. Or use YUY2. Depends what your target is.

Solution to (1): No solution! Possible improvements: either don't make the black lines totally black, and/or increase the overall brightness to compensate. Problem with the latter is that you'll clip highlights.
Yeah, brightness can be a problem when things are blended together.
I saw that when I encoded this (http://www.youtube.com/watch?v=HaDi8maw6t4) video - the game shows only half of the 360° "stream" weapon per frame, so blending two frames together (via Overlay(SelectEven, SelectOdd, opacity=0.5)) reduces the brightness. On a real screen it looks better, probably because the brighter pixels take more time to fade to black.

How can I separate/combine luma and chroma?

[Usedocne]

One of the problems with display emulation seems to be that there are a couple of different retro displays "Arcade(low-res) / CRT TV(med-res) / CRT PC(hi-res)", so imo people usually have different requirements based on what they're displaying.

Some ideas for lessening the screen darkening problem.

1. Make the scanlines or aperture grille/shadow mask semi-transparent.

2. Make the lines between the dark ones slightly brighter.

3. A modified levels curve to lighten the screen w/o clipping whites.

4. Combination of all three.

ps. Didn't NTSC CRT's have a lower/darker gamma ratio than modern displays to begin with?

[pandy]

or use ffdshow... (picture properties)

[Usedocne]

@pandy
FFDShow's sl's are a bit on the soft side imo, compared to some other avisynth methods. Also ffdshow can't do customized sl's like creaothceann was after, or aperture grille/shadow mask effects.

[pandy]

It is only matter of tweaking - overlay, scanline - both can be used on ffdshow and they are really fast due of way how ffdshow work.

[Didée]

ffdshow's scanline feature is good and all, but it can not achieve what was asked for: 4 original lines, 1 additional black line, 4 original lines, 1 additional black line, ...

If ffdshow's generic way of "scale + scanlines" is sufficient, then the (same fast) solution was already posted in #3. Just push the resize from the very end to the very start.

[pandy]

Pointresize is fast... downsizing bilinear in ffdshow also...

[Boulotaur2024]

---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
Script error: Invalid arguments to function "mt_lutspa"
(D:\test.avs, line 5)
---------------------------
OK
---------------------------

When trying to use the two creaothceann's functions.
I have both mt_masktools-26 and mt_masktools-25 (2010/9/7) in my plugins folder, any ideas ?

ps : this is the incriminated line in the .avs :
Mask = mt_lutspa(false, "y 6 % 0.01 < 0 255 ?")

[Didée]

I have both mt_masktools-26 and mt_masktools-25 (2010/9/7) in my plugins folder, any ideas ?
Remove the -26 one. Push it in a folder far, far away from Avisynth v.2.5.x.

[Boulotaur2024]

Nope, same error. I even deleted the -26 one and still the same problem.

I noticed that this line doesn't work :
mt_lutspa(false, "y 6 % 0.01 < 0 255 ?")

While this one (yours actually) does work :
mt_lutspa(relative=false,expr="y 6 % 0 == 255 0 ?")

EDIT: ok got it, I needed to name the parameters before assigning them like in the last example to make it work !

EDIT : doesn't look too bad indeed !

http://i55.tinypic.com/fyh2mo.png

[Boulotaur2024]

Here's a nice thread: http://board.byuu.org/viewtopic.php?f=10&t=147
Wow I'm really impressed by what this guy (cgwg) has been able to achieve with his emulation.
This (http://img5.imageshack.us/img5/761/sf2scanlines2.png) is probably the best old arcade CRT emulation I've ever seen. Check out the whole thread (http://nfgworld.com/mb/post/2011;?unb507sess=3c83d4f74b0874d38db6c49a2c7e0bec), I thought it was pretty interesting. Maybe someday we'll see this happen in avisynth too : )

[creaothceann]

EDIT : doesn't look too bad indeed !

http://i55.tinypic.com/fyh2mo.png

As I said, TV scanlines work a bit differently. But I have no doubt that with some tweaking you can get a visually pleasing result like that. :)


Didée: Is MT-26 for the next AviSynth version?
Speaking of which, I tried installing a version I got from SourceForge, but it didn't seem to do much... (and I don't know if filters will be compatible)

[Dogway]

I made a comparison between the script of this forum and the command line program from here (http://nfgworld.com/mb/post/2011) pointed out by Boulotaur2024 (http://forum.doom9.org/showthread.php?p=1442946#post1442946):

RAW x4
http://img267.imageshack.us/img267/2276/rawc.th.png (http://img267.imageshack.us/img267/2276/rawc.png)

CRT x4 (avs vs scanlines.exe)
http://img267.imageshack.us/img267/5503/cmdo.th.png (http://comparescreenshots.slicx.com/comparison/79925)


PD: I still dont know how to load the params.txt : /

[creaothceann]

You just add the file's name as the last parameter, I think.

...anyway, blargg has some nice NTSC filter libraries (http://slack.net/~ant/libs/ntsc.html) for some systems. (examples page (http://blargg.parodius.com/ntsc-presets/))