Attempt little improve motion compensated denoising, need help

[yup]

Hi folks!
Simple idea for improving motion compensated denoising.
First calculating motion vectors and motion compensated frames

fieldsf=prefilter(fields)
bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, delta = 2, idx = 1, overlap=4)
bv2 = fields.MVAnalyse(blksize=8, isb = true, delta = 4, idx = 1, overlap=4)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, delta = 2, idx = 1, overlap=4)
fv2 = fieldsf.MVAnalyse(blksize=8, isb = false, delta = 4, idx = 1, overlap=4)
bc1 =fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
bc2 =fields.MVCompensate(bv2, idx=2, mode=1, thSAD=16000)
fc1 =fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
fc2 =fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)


Next calculating mask
mf1=fields.MVMask(fv1,kind=1,ml=150,Ysc=255)
mf2=fields.MVMask(fv2,kind=1,ml=150,Ysc=255)
mb1=fields.MVMask(bv1,kind=1,ml=150,Ysc=255)
mb2=fields.MVMask(bv2,kind=1,ml=150,Ysc=255)

Now need choose 2 motion compensated frames from 4 available using mask value, it is problem for me, how this implement.
Last using current frame and 2 motion compensated in any temporal filer (MVDegrain1, Mediablurt, DegrainMedian, clense or other) without restriction for remove pulse noise. We will be have always 2 better motion compensated frames for filtering. This could good work at scene change.
Please advice way for implementation mask comparing.
With kind regards yup.

[g-force]

Hey yup,

first of all, you missed an "f" after "fields" in your bv2 line.

I think I'm starting to understand your question. So you want to compensate frames using temporal radius 2 and then feed the best 2 into a median type filter with temporal radius 1. Seems like a good idea so far. Instead of a mask, why not try some sort of math do determine how much a particular pixel in the compensated frames has strayed from the original, and use the 2 best to construct 2 new frames to feed into the temporal radius 1 filter. Just a thought, since nothing comes to mind on how to utilize the mask. Let me know if I'm understanding this correctly, and if you would like some help I'll try my best (which seems to be not good enough for some lately) :)

Oh, one other thing, I'm not used to working with interlaced material, so I'm wondering if something like this would be better after de-interlacing.

-G

[Didée]

Well, I see the problem, but not necessarily the solution. Just figuring the "best" compensations is doable, but rather complicated to do with a script. Also, there's no guarantee that you get the "best suited" from the point of view of what should be achieved.

First, there IS impulse noise. This noise misleads the motion search. Now, you're using some "prefilter" to handle this problem, but: this prefilter is not perfect (if it was, you would be done already). Either it doesn't remove the impulse noise completely (so the problem basically remains), or it removes it, but has unwanted side-effects which in turn make the decision insecure.

Anyhow, if impulse noise is present, then it's probably not a good idea to make a per-pixel comparison between original and compensation ... you dont't want that compensation that is the closest to the impulse noise (in case of an insufficient prefilter), and you don't want that compensation that is closest to prefilter's artefacts (in case of a bruteforce prefilter).

A quick suggestion: Simplify the task by simply taking block SADs from MVMask as criterium (like you proposed), and compairing only the (+2)-compensation with the (-2)-compensation, resp. the (+1)-compensation with the (-1)-compensation:

fieldsf=prefilter(fields)
bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, delta = 2, idx = 1, overlap=4)
bv2 = fieldsf.MVAnalyse(blksize=8, isb = true, delta = 4, idx = 1, overlap=4)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, delta = 2, idx = 1, overlap=4)
fv2 = fieldsf.MVAnalyse(blksize=8, isb = false, delta = 4, idx = 1, overlap=4)
bc1 =fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
bc2 =fields.MVCompensate(bv2, idx=2, mode=1, thSAD=16000)
fc1 =fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
fc2 =fields.MVCompensate(fv2, idx=2, mode=1, thSAD=16000)

mf1=fields.MVMask(fv1,kind=1,ml=150,Ysc=255)
mf2=fields.MVMask(fv2,kind=1,ml=150,Ysc=255)
mb1=fields.MVMask(bv1,kind=1,ml=150,Ysc=255)
mb2=fields.MVMask(bv2,kind=1,ml=150,Ysc=255)

f2_betterthan_b2 = mt_lutxy(mf2,mb2,"x y < 255 x y > 0 128 ? ?",U=3,V=3).removegrain(11)
f1_betterthan_b1 = mt_lutxy(mf1,mb1,"x y < 255 x y > 0 128 ? ?",U=3,V=3).removegrain(11)

bt2 = mt_merge( bc2, fc2, f2_betterthan_b2, U=3,V=3 )
bt1 = mt_merge( bc1, fc1, f1_betterthan_b1, U=3,V=3 )

interleave(bt2,fields,bt1)
#Clense(previous=last,next=last) # for latest version of Clense
Clense(reduceflicker=false) # for older versions of Clense
SelectEvery(3,1)

It wouldn't be a problem to make things more complicated, but I have my doubts about the possible benefits, which most probably would be rather small ...

[yup]

g-force and Didée!
Thank You for reply.
g-force I agree need:
bv2 = fieldsf.MVAnalyse(blksize=8, isb = true, delta = 4, idx = 1, overlap=4)
Also You right uderstand my idea.
Didée I make cut paste error need for mask calculation:
mf1=fieldsf.MVMask(fv1,kind=1,ml=150,Ysc=255)
mf2=fieldsf.MVMask(fv2,kind=1,ml=150,Ysc=255)
mb1=fieldsf.MVMask(bv1,kind=1,ml=150,Ysc=255)
mb2=fieldsf.MVMask(bv2,kind=1,ml=150,Ysc=255)
Not very interesting mask from source with pulse noise (I try use this for detection impulse noise but without result). I note that using motion compensated approach give better result than 2D median filter (or even 1D only vertical). I want write script which choose 1 set from (central n-1, n, n+1 forward n, n-1, n-2 and backward n, n+1, n+2). May be exist other choice (for example n-2, n, n+2) but I not discussed their. I note on frame by frame view in Virtualdub that backward or forward set give better result than central even for smooth motion scenes. This problem for only pulse noise, I agree that using MVDegrainx (x=1,2,3) for many cases simpler (MVDegrain have internal restriction thSAD which exclude bad motion compensated frames with pulse noise), but my source have pulse noise.
Didée thanks for lutxy advice (during weekend I read masktools doc).
g-force for mask You propose use RMS instead thSAD?
With kin regards yup.

[yup]

Hi!
See folowing thread:
http://forum.doom9.org/showthread.php?p=1118491#post1118491
I try improve this script.
Also see image:
http://img234.imageshack.us/img234/9103/cbwfwvz3.th.png (http://img234.imageshack.us/my.php?image=cbwfwvz3.png)
Script for this video:
LoadPlugin("VerticalCleanerS.dll")
LoadPlugin("SSEToolsS.dll")
LoadPlugin("RemoveGrainT.dll")
AVISource("seldv2.avi")
AssumeBFF()
fields=SeparateFields()
Interleaved2Planar(fields)
VerticalCleaner(mode=2,planar=true)
Planar2Interleaved()
Weave()
#fft3dfilter(sigma=16,sigma2=10,sigma3=6,sigma4=4,bt=5,bw=16,bh=16,ow=8,oh=8,interlaced=true)
fieldsf=SeparateFields()
bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
bv2 = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=2, delta = 4, idx = 1, overlap=4, dct=1,chroma=false)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
fv2 = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=2, delta = 4, idx = 1, overlap=4, dct=1,chroma=false)
bc1 =fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
bc2 =fields.MVCompensate(bv2, idx=2, mode=1, thSAD=16000)
fc1 =fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
fc2 =fields.MVCompensate(fv2, idx=2, mode=1, thSAD=16000)
#clense(Interleaved2Planar(bc1), Interleaved2Planar(fields), Interleaved2Planar(fc1),increment=0)
filtc=clense(bc1, fields,fc1,increment=0)
filtfw=clense( fields,fc1,fc2,increment=0)
filtbw=clense(bc2,bc1, fields,increment=0)
#Weave()
top=StackHorizontal(HorizontalReduceBy2(fields),HorizontalReduceBy2(filtc))
bottom=StackHorizontal(HorizontalReduceBy2(filtfw),HorizontalReduceBy2(filtbw))
StackVertical(top,bottom)
yup.

[g-force]

g-force for mask You propose use RMS instead thSAD?
With kin regards yup.

I was actually thinking of comparing them with the source on a pixel by pixel basis instead of using a mask. Kinda like this:


fieldsf = fields.TemporalSoften(4,255,255,24,2).Repair(fields,9)

bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, delta = 2, idx = 1, overlap=4)
bv2 = fieldsf.MVAnalyse(blksize=8, isb = true, delta = 4, idx = 1, overlap=4)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, delta = 2, idx = 1, overlap=4)
fv2 = fieldsf.MVAnalyse(blksize=8, isb = false, delta = 4, idx = 1, overlap=4)
bc1 = fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
bc2 = fields.MVCompensate(bv2, idx=2, mode=1, thSAD=16000)
fc1 = fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
fc2 = fields.MVCompensate(fv2, idx=2, mode=1, thSAD=16000)


best_b = Interleave(bc1,fields,bc2).Clense().SelectEvery(3,1)
best_f = Interleave(fc1,fields,fc2).Clense().SelectEvery(3,1)

Interleave(best_b,fields,best_f).Clense() #Didée's Clense might be better choice

SelectEvery(3,1)

-G

[yup]

g-force!
Thanks for support. See picture
http://img225.imageshack.us/img225/3744/3passmedgc2.th.png (http://img225.imageshack.us/my.php?image=3passmedgc2.png)
It is output Your proposed script. In my post output from backward compensation little better. For exercise I can upload my source.
yup.

[g-force]

yup,

I think it would be helpful to post your source. From your original screencap, one can tell that the current frame "fields" is bad, "filtbw" is good (which means that both bc2 and bc1 have to be good) and "filtc" is bad (so fc1 has to be bad as well). I guess what I'm getting at is that it appears that the same noise appears on multiple (at least 2) consecutive frames, which means that the noise isn't as "impulsive" as it might appear. This may require a median filter with temporal radius greater than 1. This is going to be slow, but try something like:

fieldsf = fields.TemporalSoften(4,255,255,24,2).Repair(fields,9)

bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, delta = 2, idx = 1, overlap=4)
bv2 = fieldsf.MVAnalyse(blksize=8, isb = true, delta = 4, idx = 1, overlap=4)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, delta = 2, idx = 1, overlap=4)
fv2 = fieldsf.MVAnalyse(blksize=8, isb = false, delta = 4, idx = 1, overlap=4)
bc1 = fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
bc2 = fields.MVCompensate(bv2, idx=2, mode=1, thSAD=16000)
fc1 = fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
fc2 = fields.MVCompensate(fv2, idx=2, mode=1, thSAD=16000)

Interleave(bc2,bc1,fields,fc1,fc2).MedianBlurT(0,0,0,2).SelectEvery(5,2)

-G

[Didée]

g-force: good idea! However, sorry to tell, not exactly new ... we've already gone through most of this some time ago (http://forum.doom9.org/showthread.php?t=121197). ;)

This thread here is about the idea of not just brute-force filtering over a big temporal radius, but about choosing only few frames from within the bigger radius, and use those few for filtering.

However, I still don't see the magic solution here. Sure, if you compare the filter results of those different subsets, you can see that some subsets work better than others.
But the basic problem remains: you can't get a (simple) filter to decide reliably which of the subsets is the best for a given frame. Especially in this scene with the high-contrast checker pattern, the streaks are very hard to isolate for a filter. The artefacts simply get lost in the high energy of the present texture, and even worse, the "characteristics" of the strikes are not all that different from that texture.

If you (you=>the filter) can't tell the difference between true signal and artificial signal, then you also can't decide which of the compensations are the better or the worse ones, in regard to the artefacts that you want to remove.

Full circle done, we're back at base one.

[yup]

g-force!
Didée right it is starting point for denoising this source one year ago. Thanks Didée for idea. Now I try introduce mask for better denoising. Median filtering without mask give more artifacts. First access give good result but now I want choose from 4 motion compensated frames two.
Link for source http://www.sendspace.com/file/d20ctf

See script http://forum.doom9.org/showthread.php?p=1118491#post1118491
Introduce mask give small improvement for remove pulse noise.
Idea simple find 4 motion compensated, choose 2 with small SAD or RMS, make median filter and for bad motion scene use spatial denoiser.
yup.

[2Bdecided]

yup,

I remember our previous discussion about this clip.

I bought a different VCR, and completely solved this problem for my tapes. I haven't had chance yet to investigate why it works so well - it might be because it's a better VCR, or it might be because it's worn out (just like the machine the tapes were originally recorded on)!

I'll post some screen caps when I get chance - you won't believe the difference.

Cheers,
David.

[g-force]

This thread here is about the idea of not just brute-force filtering over a big temporal radius, but about choosing only few frames from within the bigger radius, and use those few for filtering.

I'm sure yup wouldn't mind using the "brute-force" method if it gave him good results. But from a purely academic standpoint, I too would like to see something like this. And I'm not sure that either of our methods is really living up to this ideal though. You are creating new frames based on how good the block SADs are, and (in my first attempt) I am creating new frames based on how alike they are to the center frame. But really, both of these methods are just like taking four apples, cutting them up and selecting the best bits to make two new apples and then choosing which bits between those 2 and another apple are the best. Why not just cut out the middle step and choose the best bits from the 5 apples?

Okay, another idea I had to get back to the more academic exercise is to average the ylevels between the 5 frames and use only those frames which are closest in ylevel to the average. Just a thought.


If you (you=>the filter) can't tell the difference between true signal and artificial signal, then you also can't decide which of the compensations are the better or the worse ones, in regard to the artefacts that you want to remove.

Full circle done, we're back at base one.

Agreed!

-G

[yup]

Thanks all for reply.
2Bdecided which type VCR solve problem with bad VHS? I use for capture Panasonic DMR-ES35V DVD recorder with VCR, time base corrector Cypress CTB-100, Ixos S-Video cables and Pinnacle Studio MovieBoard Plus 700-PCI V.11. Before I try make capture in company specilaized on video but my result will be better. 2Bdecided advice VCR which I can buy now?
yup.

[2Bdecided]

I bought a PANASONIC NV-HS860, but like I said, it might be because it's worn out that it works so well with these tapes - I need to test further! I'm using S-VHS, but the problem is basically the same.

Cheers,
David.

[yup]

2Bdecided! :thanks: for advice choosing VCR.
May be simpler implement more weak approach:
Before scene changes use n-2, n-1 and n frames, at scene changes n, n+1, n+2 in flow motion n-1, n, n+1 and during bad motion estimation 2D median filtering.
yup.

[yup]

Didée!
Please explain what means:
"x y < 255 x y > 0 128 ? ?"
I know about RPN but can not fully understandt how work if operator.
yup.

[2Bdecided]

yup,

I've attached some screen shots showing the difference between two machines. Both machines have TBC on and DNR off. There is no sharpness setting on the 121; the sharpness setting on the 860 is set to auto. The tape is S-VHS-C camcorder original (from 1996 it seems!).

"Panasonic_NV-SV121EB.jpg" was captured into Canopus ADVC110 last year. Levels are wrong, but you can see the same problem as yours, on the time and date. The VCR was nearly new at this time (maybe ten hours use).

"Panasonic_NV-SV121EB_later.jpg" is the same tape in the same VCR, captured four months later onto DVD. The VCR had only played five more tapes, but dirt on heads made the result much worse! I cleaned the heads, and the picture went back to looking like the one in "Panasonic_NV-SV121EB.jpg"

"Panasonic_NV-HS860.jpg" is captured into Canopus ADVC110 just this weekend (so the above problems are not just tape damage). This machine is quite old and has had much more use. I haven't cleaned the heads yet. There are no problems around the time and date, but the whole picture is softer. Sharpening it does not make it quite as good as NV_SV121EB, but close - and without problems. (The images were oversharpened in the original camera - can't fix that - blinddehalo makes them far too soft).

I also tried a JVC HR-S5967EK - it also produced a softer picture, but some of the lines were still there, and the "stabiliser" in this machine didn't work as well as the TBCs in the Panasonic machines.

So, I am not saying you should try this exact VCR - just saying that maybe any different VCR is worth trying.

TBC on 860 is slightly better than on 121. Even on very good tapes, I wouldn't capture without a TBC because it makes vertical lines less wobbly - even if they look "OK" with TBC off, they look better with TBC on!

Cheers,
David.

[yup]

David!
I can not see image, please upload using http://imageshack.us/ or other upload service.
yup.

[2Bdecided]

They should be visible to you once a moderator has approved them.

I've attached the JVC one here too. What you can't see from a still picture is that the video isn't very stable with this - it's a bit jumpy.

Cheers,
David.

[yup]

David!
I see only Attachments Pending Approval. I can not download image. Please use upload services.
yup.

[2Bdecided]

They are there now.

Cheers,
David.

[yup]

David!:thanks:
Now I see difference.