Motion Compensated Denoiser...

[Soulhunter]

Interesting idea, no... (http://mf.onthanet.com/forum/thread.php?threadid=119&boardid=16&styleid=1&sid=6820ffe1dfa4558a456a0d10a29a5a26)

[Mug Funky]

hmm. that is a pretty good idea.

btw, why doesn't mf post here anymore? i must have missed something (if this is irrelevant, then you could PM me rather than offtopic'ing this thread).

i wish i could code... it looks like vaguedenoiser could be mixed in with Xvid.

but i wonder how well it would work? the theory sounds excellent, but a proof-of-concept is required i think. maybe somebody could do 2 Xvid encodes offset by 4 pixels and overlay them to see how they look?

[edit]

giving it a go now... :)

using R4 Akira special edition - this is noisy as hell and oversharp, but otherwise a very good DVD (some macroblocking due to too much high freqs)

i'm offsetting by 4 pixels horizontally and vertically, then overlaying the corrected results when i'm done encoding. it's a 500 frame clip.

[edit edit]

preliminary test didn't do much at all... certainly didn't hurt the source (after subtracting source and filetered, all i saw different was a bunch of DCT noise that presumably had been removed).

perhaps a higher quantizer should be used? i used CQ 2 and no b-frames, h-263 matrix, everything else more or less default.

maybe overlaying 4 xvids might help it?

[Wilbert]

the theory sounds excellent
XviD is not a good denoiser:

http://forum.doom9.org/showthread.php?s=&threadid=71485

[Mug Funky]

um, wilbert - aren't those tests designed for xvid as codec, not xvid as processor?

swimming noise issues would be sufficiently cancelled by the pixel offsetting i would have thought.

idea here is to use Xvid as a kind of motion compensated denoiser, rather than an end result (which could be coded in anything).

my preliminary results imply that xvid isn't all that useful at a low quant because it simply keeps too much of the original signal.

we'll see if anyone else can make some use of it, but i suspect it won't be as useful as a good deblock/dering unless it was hacked to pieces in a way that i couldn't achieve with my current l33t skillz.

[edit]

swapped "high" for "low"... i always get the quant thing mixed up.

[Manao]

Where are the script gurus I knew ? Those who would have seen that a filter allows to do motion compensation ( MVCompensate ), and that another one allows DCT coefficients decimation ( DCTFilter ).

Try that function : function mcdenoiser(clip source, float a1, float a2, float a3, float a4, float a5, float a6, float a7, float a8)
{
mvf = source.MVAnalyse(isb = false, lambda = 2000)
mc = source.MVcompensate(mvf)
diff = YV12LUTxy(source, mc, yexpr = "x y - 2 / 128 +", y=3, u = 1, v = 1)
diff2 = diff.DctFilter(a1, a2, a3, a4, a5, a6, a7, a8)
return YV12LUTxy(mc, diff2, yexpr = "y 128 - 2 * x +", y=3, u=1, v=1)
}You'll see it's not that efficient. However, you can put any denoiser you want instead of DctFilter. Go fetch the latest MVtools (http://jourdan.madism.org/~manao/MVTools-v0.9.5.3.zip), you'll then even be able to filter chroma.

[Mug Funky]

thanks, manao. i forgot about DCTfilter.

i'd like something like the SPP deblocking in mplayer - AFAIK it quantizes and offsets each block and sums them all together. it takes ages, but looks pretty good (little too blurry, and weird things happen to edges).

but yeah, something tells me that codec-as-denoiser isn't going to be a very elegant way of doing things - that after all is what inloop filtering is for.

[edit]

while you're here, manao:

i get an access violation 10 frames in when using this function (just a simple smudge-then-mask denoise)

function mvAnal (clip c, int "lambda", int "blocksize", int "searchtype", int "stepsize", int "pel")
{

lambda = default(lambda,2000)
blocksize = default(blocksize,8)
searchtype = default(searchtype,2)
stepsize = default(stepsize,3)
pel = default(pel,1)

global vb=c.mvanalyse(lambda=lambda, sx=blocksize, sy=blocksize, stp=stepsize, pel=pel, isb=true, st=searchtype)
global vf=c.mvanalyse(lambda=lambda, sx=blocksize, sy=blocksize, stp=stepsize, pel=pel, isb=false, st=searchtype)

return c

}

function mvClean (clip c)
{

c.mvanal()

cfw = c.MVcompensate(vf)
cbw = c.MVcompensate(vb)

comp = yv12lutxy(cfw,cbw,Yexpr="x y + 2 /")
smudge = yv12lutxy(c,comp,Yexpr="x 3 / y 2 3 / * +")

cmask = yv12lutxy(c,smudge,Yexpr="x y - abs",Y=3,U=-128,V=-128).inflate().levels(3,1,5,0,255,coring=false)


maskedmerge(smudge,c,cmask)
}

mvanal is just a shortcut i use to get forward and backward vetcors when i can't be bothered typing so much :)

other plugins i'm using are MPEGsource (nic's new one) and tritical's new Tdeint


[edit 2]

adding ".crop(16,16,-16,-16,align=true)" after Tdeint fixed it. must be a memory alignment thing (i don't understand optimizations at all :()

[Soulhunter]

Originally posted by Mug Funky

btw, why doesn't mf post here anymore? i must have missed something (if this is irrelevant, then you could PM me rather than offtopic'ing this thread).

He was banned... :(


Bye

[Manao]

mug funky : no, there's still a bug in mvcompensate. I'll try to find and correct it tonight.

[Mug Funky]

update:

i tried averaging 4 xvids instead of 2, and i got some good results.

i haven't done comparisons with other denoisers yet, but basically i got 84% of the bitrate compared to an un-filtered encode. that's significant i think.

as far as visual quality goes, it looks rather like a deblocked video, except the grain is almost completely gone, as is a lot of the original DCT noise. there's some flickering blocks in the background, which i suppose i could get rid of with removedirt (and get a slight increase in compressibility too).

my test clips are 18-24 megs, so i won't be posting them unless someone really wants it.

here's my script though... unrem the appropriate lines and you might make sense of it. it is a bit ad-hoc and thrown together.


MPEG2Source("I:\AKIRA\VIDEO_TS\VTS_04_1 - 0xE0 - Video - MPEG-2 - 720x576 (PAL) - 16~9 - Letterboxed.d2V")

trim(27000,27000+500)

bicubicresize(640,480,0,.5)
#undot()
#AddGrain(20,.2,.2)

#mvClean()

#carefuldirt()

c=last
c

#d = c.addborders(2,2,0,0)
#TL = d.addborders(0,0,2,2).mod16()#.DCTsmudge().crop(0,0,-2,-2).mod16()
#TR = d.addborders(2,0,0,2).mod16()#.DCTsmudge().crop(2,0,0,-2).mod16()
#BL = d.addborders(0,2,2,0).mod16()#.DCTsmudge().crop(0,2,-2,0).mod16()
#BR = d.addborders(2,2,0,0).mod16()#.DCTsmudge().crop(2,2,0,0).mod16()

#TL
#TR
#BL
#BR

#TL = avisource("I:\AKIRA\VIDEO_TS\TL.avi").crop(0,0,-2,-2).mod16()
#TR = avisource("I:\AKIRA\VIDEO_TS\TR.avi").crop(2,0,0,-2).mod16()
#BL = avisource("I:\AKIRA\VIDEO_TS\BL.avi").crop(0,2,-2,0).mod16()
#BR = avisource("I:\AKIRA\VIDEO_TS\BR.avi").crop(2,2,0,0).mod16()

#top = yv12lutxy(TL,TR,Yexpr="x y + 2 /",Uexpr="x y + 2 /",Vexpr="x y + 2 /",Y=3,U=3,V=3)
#bottom = yv12lutxy(BL,BR,Yexpr="x y + 2 /",Uexpr="x y + 2 /",Vexpr="x y + 2 /",Y=3,U=3,V=3)
#yv12lutxy(top,bottom,Yexpr="x y + 2 /",Uexpr="x y + 2 /",Vexpr="x y + 2 /",Y=3,U=3,V=3)
#crop(2,2,-14,-14,align=true)

#filtered = avisource("I:\AKIRA\VIDEO_TS\testclip_comp_4x.avi")

#subtract(c,filtered)

[edit]

by the way, shouldn't this be in the usage section?

[Soulhunter]

Originally posted by Mug Funky

my test clips are 18-24 megs, so i won't be posting them unless someone really wants it.

Maybe 1-3 before/after pictures would do it... :)


Tia n' Bye

[morsa]

May be in conjunction with DNR?
Does anybody know if it has been improved?


@Soulhunter
So, will mf be able to post his scripts here again?
It's been a big loss......

[Soulhunter]

Originally posted by morsa

@Soulhunter
So, will mf be able to post his scripts here again?
It's been a big loss......

Yes, users will miss all his new 1fps scripts... :(


Bye

[Manao]

mug funky : can you make available a 7 frames clip which show the crash you're experiencing ? Because if I was able to reproduce it at work, at home I can't :(

[Mug Funky]

unfortunately it's one of those elusive crashes that happen after several f5/f7 reloads in virtualdubmod.

restarting the program fresh often helps, but sometimes it's just a matter of the lunar cycle...

[Manao]

Mug Funky : In that case, I don't know if I'll be able to find what the bug is. You can try the latest build I made ( description (http://forum.doom9.org/showthread.php?s=&postid=546948#post546948) ), it might solve your issue.

@all : have a look at the latest version, there's a filter in it which might interest you ( EncDenoise ).

[Mug Funky]

thanks for the new functions, manao. very cool idea, which i'm just testing a little.

i won't bother you with my neurotic computer until i can make the bug repeatable... it's still there but it comes and goes rather frustratingly (sort of like how your car always starts running properly when you take it to a mechanic, or your persistent cough goes away 10 minutes before you see the doctor...).