Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
4th September 2011, 12:18 | #1 | Link |
Registered User
Join Date: Jul 2010
Location: Germany
Posts: 357
|
60 -> 30 fps flickering with minimum blur
Over at tasvideos.org we're in a specific situation: videogame emulator recordings are 60fps, but Youtube allows only 30fps clips. So the first solution would be to drop every other frame. Unfortunately many games simulate transparency by turning off layers every other frame (e.g. when the player's sprite was hit by an enemy). On YT this shows up as either fully transparent or opaque sprites, depending on if the first frame was even or odd.
After some time we came up with TASBlend (script and 60fps example clip). It keeps the flickering; the problem is a slight blurring of the whole screen. So... Maybe we could somehow filter out the pixels that are the same color every other frame, but other colors in-between; the resulting mask could then be used to overlay the TASBlend-ed clip only to the detected (flickering) regions? |
4th September 2011, 19:26 | #2 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
Edit: I originally put a solution up that only worked by making a particular assumption, but it wasn't robust in general. This version is more in line with the OP's request:
Code:
function TASBlend(clip c, float "ratio" ) { # reduces framerate to 1/2 but leaves flicker effects partly visible # blends frame pairs with alternating opacity (default is 2/3+1/3;1/3+2/3) # optional "ratio" is the opacity of the first frame out of the four ratio = default( ratio, 2.0/3 ) opacity = round( ratio * 257 ) # Blend flicker to half-rate c.SelectEvery( 4, 1,0,2,3 ) blend = Layer( SelectEven(), SelectOdd(), level=opacity ) # Determine flicker is a simplistic way - compare each pixel with the previous and next two frames Similar2 = 1 # How similar current pixel must be to the same pixel 2 frames forwards and backwards Different1 = 8 # How different current pixel must be from the same pixel 1 frame forwards and backwards maskf2 = Overlay( c, c.SelectEvery(1, 2), mode="difference", pc_range=true ).Greyscale().Levels( 128+Similar2, 1.0,129+Similar2, 0,255, false ).Invert() maskf1 = Overlay( c, c.SelectEvery(1, 1), mode="difference", pc_range=true ).Greyscale().Levels( 128+Different1,1.0,129+Different1, 0,255, false ) maskb1 = Overlay( c, c.SelectEvery(1,-1), mode="difference", pc_range=true ).Greyscale().Levels( 128+Different1,1.0,129+Different1, 0,255, false ) maskb2 = Overlay( c, c.SelectEvery(1,-2), mode="difference", pc_range=true ).Greyscale().Levels( 128+Similar2, 1.0,129+Similar2, 0,255, false ).Invert() maskf = Overlay( maskf1, maskf2, mode="multiply", pc_range=true ) maskb = Overlay( maskb1, maskb2, mode="multiply", pc_range=true ) mask = Overlay( maskf, maskb, mode="multiply", pc_range=true ).SelectEven() # Only use blend where detected flicker Layer( c.SelectEven(), blend.Mask(mask) ) } I'm sure there's a better way than all those Overlays & Levels, perhaps some TemporalSoften trickery, or convert to YUV and use MaskTools instead (since Overlay converts to YUV anyway), but I put this together quickly to correct my earlier mistake... Last edited by -Vit-; 4th September 2011 at 21:41. Reason: tweak |
5th September 2011, 13:43 | #4 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
You can still use Masktools on RGB with this simple preprocessing step:
Code:
function RGBintoYV12(clip v) { #place r,g,b data directly into u,y,v planes #this result has no visual meaning but is only for achieving arithmetic on the data v pointresize(width*2,height*2)#doublesize to allow full 4:4:4 color resolution with u,v r=ShowRed.converttoyv12 g=ShowGreen.converttoyv12 g=g.pointresize(width*2,height*2)#this is destined for Y which must be bigger b=ShowBlue.converttoyv12 YtoUV(r,b,g) } |
5th September 2011, 16:47 | #5 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
creaothceann: There is no "difference" mode in Layer afaik. There is "subtract", but that does something rather different. However, Overlay is only used to build the mask.
BTW the algorithm you proposed is reasonably effective, but has limitations. I already noted the issue with moving flickering objects - the pixels aren't in the same place frame to frame so the flicker is not detected. Also flickering objects that are also animating or changing color at the same time - individual pixels may be briefly mis-detected as not flickering because the color changes "cancel out" the flicker temporarily. jmac698: Something like that + some masktools may well be more efficient. Though efficiency is not a major issue on such a small resolution. Last edited by -Vit-; 5th September 2011 at 17:34. |
5th September 2011, 21:47 | #6 | Link |
Registered User
Join Date: Jul 2010
Location: Germany
Posts: 357
|
Yeah, I was afraid of that... oh well. My only idea would be to increase the detected area somehow to compensate for moving sprites.
Anyway, here's some new code that shows the best results so far: http://tasvideos.org/forum/viewtopic...=286117#286117 |
5th September 2011, 22:06 | #7 | Link | |
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,496
|
Quote:
David |
|
6th September 2011, 23:23 | #9 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
I see on the TAS site you are trying MVTools approaches. Below is a refined version of that. It works very well on the sample you gave above, but it may yet need tweaking. However, in general I would expect problems with any MVTools approach where there are fast animations or detailed backgrounds. The "Sensitivity" setting in the code might help, but... [Edit: I just tried it on the Roadrunner clip you have on the TAS site, a difficult example, had to cut the Sensitivity value to the range 0 to 20 for best results]
Code:
PointResize( Width()*2, Height()*2 ) ConvertToYV12() # Add matrix="PC.601" if you intend to convert back to RGB at the end # How accurately to identify flicker. Increase if flicker is being lost, but with # the increased chance of distortion or ghosting. Or decrease to reduce distortion # and ghosting but with potential to miss flicker Sensitivity = 75 clip = last clipOdd = clip.SelectOdd() w = clip.Width() h = clip.Height() srchClip = clipOdd.RemoveGrain(12).GaussResize( w,h, 0,0, w+0.0001,h+0.0001, p=2 ).Merge( clipOdd, 0.1 ) srchSuper = srchClip.MSuper( pel=1 ) fVec = srchSuper.MAnalyse( isb=false, blksize=16, overlap=4, truemotion=false ) bVec = srchSuper.MAnalyse( isb=true, blksize=16, overlap=4, truemotion=false ) fVecR = srchSuper.MRecalculate( fVec, blksize=8, overlap=2, truemotion=false ) bVecR = srchSuper.MRecalculate( bVec, blksize=8, overlap=2, truemotion=false ) oddSuper = clipOdd.MSuper( pel=1 ) newOdd = clipOdd.MFlowFPS( oddSuper, bVecR, fVecR, num=0, mask=2, ml=50 ) # Could tweak the ml value here too... maskb = MMask( clipOdd, bVecR, kind=1, ml=Sensitivity+0.1 ) maskf = MMask( clipOdd, fVecR, kind=1, ml=Sensitivity+0.1 ) mask = mt_logic( maskb.SelectEven(), maskf.SelectOdd(), "max", U=-128,V=-128 ) Interleave( clip.SelectEvery(4,0), mt_merge( newOdd.SelectEvery(4,1), clip.SelectEvery(4,2), mask, luma=true ) ) # Add these lines to restore original sized RGB (chromaresample requires AviSynth 2.6) #ConvertToRGB( matrix="PC.601", chromaresample="point" ) #PointResize( Width()/2, Height()/2 ) Last edited by -Vit-; 7th September 2011 at 01:47. |
7th September 2011, 10:08 | #10 | Link |
Registered User
Join Date: Dec 2002
Location: UK
Posts: 1,673
|
This is a real challenge.
The perfectionist does the best they can on YouTube, but provides a link to a proper 60fps download at the top of the YouTube description. Mediafire seems to work well for this. Cheers, David. |
|
|