Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th March 2004, 17:07   #1  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
new deblocker script: funkydeblock

a recent thread got me thinking of deblocking, so i made one based on blindPP and high/lowpass separating (what's a generic word for that?).

it gives VERY similar results to blindPP(quant=x,cpu=4) but can eliminate some bitrate pumping effects as well, an delivers a not-insignificant compressibility gain (not as much as removedirt, but hey, we can't all be geniuses)

this is of most use on HEAVILY compressed source material, otherwise regular blindPP or mpeg2dec3's "cpu=4" will be enough.

Code:
function gauss (clip c,int "radius") {

radius=default(radius,1)

radius=radius*4
c.bilinearresize(4*(c.width/radius),4*(c.height/radius)).bicubicresize(c.width,c.height,1,0)

}

function funkydeblock (clip c,int "quant",int "th",int "radius", bool "deblock",bool "depump"){

quant=default(quant,4)
th=default(th,3)
radius=default(radius,4)
deblock=default(deblock,true)
depump=default(depump,true)

blurd = c.gauss(4)

highpass=subtract(blurd,c)
deblocked=(deblock==true) ? highpass.blindpp(quant=floor(quant),cpu=4) : highpass

nopump=subtract(blurd,deblocked)
pump=subtract(blurd.temporalsoften(radius,th,th,2*th,2),deblocked)

(depump==true) ? pump : nopump

}
the first little function is useful as well - it'll use blurry-bicubic resizing to give a gaussian-like blur at whatever radius you tell it.


okay... usage:

funkydeblock(clip c,int "quant",int "th",int "radius", bool "deblock",bool "depump")

quant: the quant param usually fed into blindPP. default 4. tis actually is very conservative due to the fact it's only used on the highpassed clip, so even quant=31 gives a good picture.

th: temporalsoften threshold. only active when depumping is on. default 3. this param controls how sensitive to pumping the filter is. keep it low, unless you're seeing large jumps at I frames. increase only as much as you have to, or dim moving objects will start vanishing.

radius: the number of frames before and after currentframe used in temporalsoften. default 4. this is intended to smear GOP changes, so pumping doesn't show as much.

deblock: turn deblocking on/off (only depumping is used). default true

depump: turn depumping on/off (only deblocking is used). default true
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 10th March 2004, 19:30   #2  |  Link
mf
 
mf's Avatar
 
Join Date: Jan 2002
Posts: 1,729
Re: new deblocker script: funkydeblock

Quote:
Originally posted by Mug Funky
high/lowpass separating (what's a generic word for that?).
Frequency discrimination?
mf is offline   Reply With Quote
Old 10th March 2004, 20:32   #3  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Cool, maybe this will help me with "The Simpsons". I was just using MipSmooth but the blocks on the source don't seem to be macroblocking (they are too small and appear mostly on flat, evenly-colored areas). How fast is this filter?


P.S: now I only need something to reduce the sudden brightness changes...
Chainmax is offline   Reply With Quote
Old 11th March 2004, 02:39   #4  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
I'll try to find a frame that shows clearly what I'm talking about.
Chainmax is offline   Reply With Quote
Old 11th March 2004, 07:17   #5  |  Link
DarkNite
Almost Silent Member
 
DarkNite's Avatar
 
Join Date: Jun 2002
Location: Purgatory
Posts: 273
After a few tests all I can really say is:

Wow, this works pretty damn good. Thank you.
__________________
Rethinking the "Why?" chromosome.
DarkNite is offline   Reply With Quote
Old 11th March 2004, 09:10   #6  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
Quote:
Frequency discrimination?
i hesitate to call it that because it's not true lowpassing... for all intents and purposes it is, but mathematically it's a huge botch

besides, you can do this with anything, not just blurring. i guess "differential filtering" but that's nondescript and probably wrong.

oh, and mf: would you have any idea how to get de-ringing going without touching the true edges too much? (and running at over 5fps :P) i did test clips for this where i encoded footage at q8 in xvid, I-frame distance 15 (so like a bad DVD), and this killed the blocks and the large scale pumping, but the ringing and pumpy-ringing remained.

i'll try "HQdering", but my computer's quite slow, hence my trying to keep scripts fast and simple (hehe. that and i don't have the short-term memory to write a "toad licking" script)
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 11th March 2004, 12:40   #7  |  Link
mf
 
mf's Avatar
 
Join Date: Jan 2002
Posts: 1,729
Look at HQDering . It removes the true edges from the thickened "around-the-edges" mask. I will have to set this functionality togglable in the next version though, cause it's crippling my HQUnfilter functionality that I now use it for the most.
mf is offline   Reply With Quote
Old 12th March 2004, 20:29   #8  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Here's a clip which perfectly illustrates the blocking issue I was talking about:

Sample.

Do you guys think funkydeblock can help clear out those blocks?

If you look closely, you'll also notice that the brightness changes slightly at a couple of spots. This is much more evident in other scenes but I couldn't locate them yet. What can I use to reduce/eliminate that?
Chainmax is offline   Reply With Quote
Old 12th March 2004, 23:04   #9  |  Link
mf
 
mf's Avatar
 
Join Date: Jan 2002
Posts: 1,729
That's not just blockiness. That's film grain, a bad transfer, denoising, bad authoring, all of which lead to a certain kind of blockiness, and also the brightness changes. I think you're the best person to try it out - the script is posted and you have the source video. Why would we have to try it out to tell you if it works or not .
mf is offline   Reply With Quote
Old 13th March 2004, 21:16   #10  |  Link
Prettz
easily bamboozled user
 
Prettz's Avatar
 
Join Date: Sep 2002
Location: Atlanta
Posts: 373
Quote:
Originally posted by Chainmax

If you look closely, you'll also notice that the brightness changes slightly at a couple of spots. This is much more evident in other scenes but I couldn't locate them yet. What can I use to reduce/eliminate that?
Well if you don't particularly value your time you could always apply ColorYUV or LumaFilter manually to every affected scene.
Prettz is offline   Reply With Quote
Old 15th March 2004, 06:50   #11  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
@chainmax:

hell yeah this deblocker can get rid of it.

just tried it then. luma variance can be treated as pumping, so you can use the script at it's default and they'll all go away

of course, it can't help with the other problems with this sample that mf mentioned, but that should be allright.

make sure you bob first (losslessly, ie with no blurring)

so you'll have:

Code:
mpeg2Source("C:\Ack\Clip1.d2v")

telecide(order=0,guide=1)

coloryuv(levels="pc->tv")
funkydeblock(th=5)
coloryuv(levels="tv->pc")

the 0,.5 in bob makes the bicubic filter work without blurring, which is important for putting the stream back together without loss.

[edit]

oh, what am i doing! you can telecide it first and save all the bobbing and un-bobbing...

also there seems to be some overbright things which invoke the subtract inverted dot bug, so the script above has been modified accordingly.
__________________
sucking the life out of your videos since 2004

Last edited by Mug Funky; 15th March 2004 at 06:59.
Mug Funky is offline   Reply With Quote
Old 15th March 2004, 09:32   #12  |  Link
mf
 
mf's Avatar
 
Join Date: Jan 2002
Posts: 1,729
Quote:
Originally posted by Mug Funky
also there seems to be some overbright things which invoke the subtract inverted dot bug, so the script above has been modified accordingly.
Why don't you workaround that in your script, maybe a togglable switch? Could save people who don't know about the bug some headaches . I've already described to you how to do it.
mf is offline   Reply With Quote
Old 15th March 2004, 11:53   #13  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
hmm... i'll maybe do that, but the bug isn't always there, and i'm wanting to nut it out properly before i use a slower workaround.

for example, the bug doesn't show up with the following:

Code:
blurred = clip.magicalblurfilter(8)

subtract(blurred,clip)
subtract(blurred,last)
but it will for:

Code:
blurred = clip.magicalblurfilter(8)

subtract(clip,blurred)
subtract(last,blurred)
which otherwise will give the same output.

but then again, i've made the deblocker by those rules (unless i got confused) but the bug shows up either way i arrange things. until i solve this i won't bother putting any extra stuff in... but if it defeats me completely then i'll do it
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 15th March 2004, 12:11   #14  |  Link
mf
 
mf's Avatar
 
Join Date: Jan 2002
Posts: 1,729
That gave me an idea. How about trying this:
ColorYUV(levels="pc->tv").Invert().ColorYUV(levels="pc->tv")
So that the padding is always 20 values off, instead of 16 off on the left and 20 off of the right. Alternative could be:
Levels(20, 1, 235, 20, 235, false)
mf is offline   Reply With Quote
Old 15th March 2004, 12:14   #15  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,855
Just a warning with subtract : on the luma, when both clip are identical, it returns 126, not 128, so it is slightly biased. It explains the dots.
Manao is offline   Reply With Quote
Old 15th March 2004, 12:53   #16  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
ah, is that it? does this mean it's a simple matter to fix?
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 15th March 2004, 13:01   #17  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,855
May be, I can't be sure of it ( I can't yet compile AviSynth on my computer ). I sent a workaround to sh0dan, but the probleme is that subtract will then have a different behavior in RGB and in YUY2. So there is also a decision to take.

If you don't need YV12LUT, I advise you to use YV12Subtract, bundled with the masktools ( it's faster than subtract ). If you use YV12LUT, you'll have to wait for me to find the bing between YV12LUT and YV12Subtract.
Manao is offline   Reply With Quote
Old 15th March 2004, 13:11   #18  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
ah. i was just about to ask if YV12subtract exhibited the same behaviour...

i'll switch to that then.
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 15th March 2004, 17:57   #19  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
Quote:
Originally posted by mf:
That's not just blockiness. That's film grain, a bad transfer, denoising, bad authoring, all of which lead to a certain kind of blockiness, and also the brightness changes. I think you're the best person to try it out - the script is posted and you have the source video. Why would we have to try it out to tell you if it works or not
I phrased myself wrong. I don't know much about blocking other than it's bad, so I wanted to know if what I call "blocks" were actually blocks and could therefore be removed with this. Sorry about that.

mugfunky: the script I'd have to use would be

IVTC
coloryuv(levels="pc->tv")
funkydeblock(th=5)
coloryuv(levels="tv->pc")
whatever else


then?
Chainmax is offline   Reply With Quote
Old 15th March 2004, 18:16   #20  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,855
I should have checked the code before answering. YV12Subtract will behave in the same way Subtract does considering the dot issue. But the value will be centered on 128.

I'll add a parameter to change that behavior. I'll also correct a slight bug that sneak its way in when resolution are mod 4 only. So wait for another release.
Manao is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:55.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.