finer control over brightness controlled filtering

[check]

Hi all,

I'm currently filtering a movie (Final Fantasy Spirits Within from Blu Ray), which has a lot of of pretty standard mucky noise. It's killed pretty effectively with fft3dfilter, except that is also destroying all the low detail shadows in the film -- which is a lot of the total movie :p

I've setup some basic filtering based on the luma of each pixel: darker pixels are filtered less.
#noise removal: filter luma least in dark areas to prevent fft3dfilter eating shadows
mt_merge(last.removegrain(3).removegrain(2), last.fft3dfilter(sigma=10).removegrain(2), last, true)


Is it possible to have finer control over the merge than just the current linear mapping? I would like to change the mask merge so that only very dark areas (say under 75) really have fft3dfilter() blended out (and everything under 30 or so should not have any fft3d at all).

[Didée]

That's no problem, just tweak the mask to be like you would like to have it.

-> Example (http://forum.doom9.org/showthread.php?t=128864).

[foxyshadis]

You can build your own merge by using mt_lutxy. You just need to figure out postfix notation to get it written.

[check]

Thanks, works well :-), and easier than I hoped. The script is eating my RAM for breakfast though :o
LoadPlugin("DGAVCDecode.dll")

AVCSource("Final Fantasy - The Spirits Within Blu-Ray\BDMV\STREAM\bits0001.dga")
crop(8,26,-6,-24)
spline36resize(1904,1024)
levels(8,1,255,0,255,True)

fft3dfilter(sigma=1, plane=4,ncpu=1)
removegrain(3)
first = last

a = last.hqdn3d(cs=2,ct=1.5)
b = last.fft3dfilter(sigma=5, plane=3,ncpu=1)
th_low = 20
th_high = 50
dmask = b.levels(th_low,1.0,th_high,255,0,false)
a.mt_merge(b,dmask,U=3,V=3,luma=true)

mt_merge(first.fft3dfilter(sigma=5, plane=3,ncpu=1), last.hqdn3d(cs=2,ct=1.5))

gradfun2db()

seesaw(first, last, bias=40)

removegrain(1)

[Didée]

@ check: Two things that seem dodgy in that script.

levels(8,1,255,0,255,True)
That line clamps all luma <24 to 16. Is the original source really too bright by so much? No useful information in luma range 16 - 24 ?


dmask = b.levels(th_low,1.0,th_high,255,0,false)

a.mt_merge(b,dmask,U=3,V=3,luma=true)

mt_merge(first.fft3dfilter(sigma=5, plane=3,ncpu=1), last.hqdn3d(cs=2,ct=1.5))
That's a valid processing chain, but I don't see the reasoning for what it does.
Note that it's actually doing this:

dmask = b.levels(th_low,1.0,th_high,255,0,false)

a .mt_merge( b, dmask,U=3,V=3, luma=true)

mt_merge( LAST, first.fft3dfilter(sigma=5, plane=3,ncpu=1), last.hqdn3d(cs=2,ct=1.5))
i.e. in the 3rd line, mt_merge takes the result of the 2nd line as first input, the 1st argument in line 3 is taken as 2nd input, and the 2nd argument in line 3 is used as mask to merge those two.

From the logic of what to apply where and why, more reasonable seems this:

dmask = b.levels(th_low,1.0,th_high,255,0,false)

a.mt_merge( b, dmask, U=3,V=3, luma=true)

mt_merge( first.fft3dfilter(sigma=5, plane=3,ncpu=1), last.hqdn3d(cs=2,ct=1.5), dmask)

And while we're at it, there is one FFT3DFilter executed twice, on the same base clip with the same settings. One of them is superfluous.

first = last

a = last.hqdn3d(cs=2,ct=1.5)
b = last.fft3dfilter(sigma=5, plane=3,ncpu=1)
[...]
dmask = b.levels(th_low,1.0,th_high,255,0,false)

a.mt_merge( b, dmask, U=3,V=3, luma=true)

mt_merge( first.fft3dfilter(sigma=5, plane=3,ncpu=1), last.hqdn3d(cs=2,ct=1.5))

But when replacing the unneeded fft3dfilter appropriately, then it looks strange in another way:

dmask = b.levels(th_low,1.0,th_high,255,0,false)

mt_merge( a, b, dmask, U=3,V=3,luma=true)

mt_merge( b, last.hqdn3d(cs=2,ct=1.5), dmask)

This means for one that you're criss-cross merging over b by dmask is happening, and also hqdn3d is processing a clip that already has been processed by hqdn3d.


A second thought about what should be applied where and why ? I'm not sure whether this is perfectly obvious.

[check]

Yes, the cold hard glare of the morning is making me think "wtf" too about all the masktools stuff.
That line clamps all luma <24 to 16. Is the original source really too bright by so much? No useful information in luma range 16 - 24 ?
The video has no luma detail from 0-8, ie the darkest black is 8. I made up that line after a quick read of the documentation to fix that.

All the masktools stuff
It's probably best to explain what I want to do:
I'd like to run a luma plane fft3dfilter over the whole video, but have the strength of it drop off under 50 luma and to not run at all under 20 luma. As a replacement to the lack of fft at the lower levels, I'd like to run some weak alternate filtering (hqdn3d + removegrain).

I'd then like to run some chroma filtering with fft3dfilter going the other way, ie have the filter drop off as the luma gets brighter.

Before both of these is some basic denoising, and after these comes the deband/sharpening.

I'm aiming to get something like this:
LoadPlugin("DGAVCDecode.dll")
AVCSource("Final Fantasy - The Spirits Within Blu-Ray\BDMV\STREAM\bits0001.dga")
crop(8,26,-6,-24)
spline36resize(1904,1024)
levels(8,1,255,0,255,True) #wrong?

fft3dfilter(sigma=1, plane=4,ncpu=1)
removegrain(3)
first = last

#filter luma, but less filtering in the dark areas

#filter chroma, but less filtering in the bright areas

#final cleanup
gradfun2db()
seesaw(first, last, bias=40)
removegrain(1)


PS: can you suggest any good ways to bias seesaw into sharpening flatter areas more strongly & more detailed areas less strongly (than the standard)? I don't want to completely reverse the polarity, but I would like to test just biasing it a little in this direction.

[Didée]

Your call levels(8,1.0,255,0,255,[coring=]true) will first scale [8,255] to [0,255], then it will clamp the range [0,16] to [16] and the range [235,255] to [235].
In effect, all original luma 0..24 will become '16' and all original luma 236..255 will become '235'. So, in particular in the dark parts you're probably losing much of the actual content.
If your source has useful information in range <16 and/or >235, you want to use "false" in the levels() call. "true" will always clamp to 16,235 at the end.


can you suggest any good ways to bias seesaw into sharpening flatter areas more strongly & more detailed areas less strongly
Not really. That's a "many ways lead to Rome" quest, and since I did't figure yet which way pleases me most, no way is open. SeeSaw only knows about single pixel's extremity in a 3x3 area. That doesn't say much about whether the actual area is "flat" or "detailed".
The closest is to activate the "# head = head.mt_merge(..." line in the script, which will disable overshoot in areas with prominent edges.
Also, I'm not sure about SeeSaw's behaviour on full-HD sources. Can't say if all those filters which work on 3x3 areas will bite strong enough on such big resolutions - it depends on the sharpness of the source. If a 3x3 median filter doesn't cut much, then SeeSaw loses its 'speciality' and becomes somewhat similar to LimitedSharpen.

* * *

I started to analyse your script in detail, but it gets too lengthy for this year.

Altered to do straightly what you described, it looks like

# corrected errors in blue

spline36resize(1904,1024)
levels(8,1,255,0,255,False)

first = fft3dfilter(sigma=1, plane=4,ncpu=1)

dark = last.removegrain(3).hqdn3d(cs=2,ct=1.5)
bright = last.fft3dfilter(sigma=5, plane=0,ncpu=1) # isn't sigma=5 too much??
th_low = 20
th_high = 50
dmask = bright.levels(th_low,1.0,th_high,255,0,false)
bright.mt_merge(dark,dmask,U=3,V=3,luma=true) # bright/dark were swapped

mt_merge(last.fft3dfilter(sigma=5, plane=3,ncpu=1), last, dmask, luma=true, Y=2,U=3,V=3)

gradfun2db()

seesaw(first, last, bias=40)

removegrain(1)

However in this way, it's a bit strange to use fft3d as initial filter, since fft3d is called two more times anyways, for both luma and chroma. That's not elegant and somehow seems like a waste.

Also, remember that SeeSaw with default settings will limit all your denoising work to be not more than +/-2 from the original pixel value. Which might be too little.


Wish a good new year to y'all, I'm leaving now to party. ;)

[check]

When I woke up this morning I had a drunken thanks typed out and ready to post, but you'll have to settle for just a hungover thanks: thanks for your help both here and in threads gone by. :-)

However in this way, it's a bit strange to use fft3d as initial filter, since fft3d is called two more times anyways, for both luma and chroma. That's not elegant and somehow seems like a waste.

Also, I'm not sure about SeeSaw's behaviour on full-HD sources.
fft3dfilter is a little like removegrain in that you can end up with better results from chaining a few calls than a single large one. This is mostly because it eats 'flat' for breakfast at higher strengths.
Here's a quick sample (60mbish): http://bluebottle.net.au/ffsw_sample.mkv
avcsource("bits0001.dga")
crop(8,26,-6,-24).spline36resize(1904,1024)
selectrangeevery(600,6)
trim(0,250)



From your script:
dmask = b.levels(th_low,1.0,th_high,255,0,false)
Meant to be this right?
dmask = first.levels(th_low,1.0,th_high,255,0,false)

I think dark and bright are backwards as well?
dark = last.removegrain(3).hqdn3d(cs=2,ct=1.5)
bright = last.fft3dfilter(sigma=5, plane=0,ncpu=1)
th_low = 20
th_high = 50
dmask = b.levels(th_low,1.0,th_high,255,0,false)
dark.mt_merge(bright,dmask,U=3,V=3,luma=true)

I'm encoding a few tests now, I'll report back once done.