Using Avisynth to help compressing files

[Razorholt]

I was wondering, which filters may be used to help compressing a good quality file (not anime)?

undot()
Fluxsmooth()

Good start?



Thanks,
- Dan

[sh0dan]

Probably the best filter: BicubicResize() to a smaller size.

Otherwise have a look at this thread (http://forum.doom9.org/showthread.php?t=51181).

[Razorholt]

Thanks shOdan.

- Dan

[Elic]

Razorholt
I shoudn't advise to use Undot() for improving compressability of hi-quality video because it eliminates small fast-moving details (like rain-drops or snow-flakes) almost completely. Instead, I prefer to use light temporal denoising likeTemporalsoften(2,3,3,mode=2,scenechange=6)
FluxSmoothST(5,7)(I stealed this :) from scripts generated by GordianKnot).

Also, light blur() (with strength of 0.15 or so) before BicubicResize() might gain the compressability, but it highly depends on source video.

[Mug Funky]

i've never seen undot produce an artefact...?

removedirt can be a bit agressive as far as removing objects rather than just noise, but for compressibility gain versus detail it can't be beaten. if used with motion compensation (especially the new stuff being developed lately) it can give a practically artefact-free compressibility boost.

degrainmedian is also very good - doesn't give blocks like removedirt can if not used judiciously, and still gives a handy compressibility gain.

[IanB]

Perhaps the new GaussResize(), adjust P to select the right amount of blur

[Didée]

May I hop in with a small function? :)

SPresso() is one I use a lot (mostly for internal processing within more complex functions), but it also deals very well as a standalone.

(It doesn't matter if you like caffeine or not (I do) - SPresso stands for Spatial Pressdown.)

On my usual, usual-noisy DVB sources, compression gain usually is from 2% to 3% (light settings -> changes almost invisible) up to 10 to 12% (stronger settings -> slight, gentle softening, not very obvious).

Format:

SPresso( limit, bias, [RGmode] )

"limit" tells the maximum change to do to any pixel
"bias" is, well, something like "aggessivity": '20' is a very light setting, '33' is already quite strong.
"RGmode" tells the RemoveGrain mode to use. The default of "4" is the best in most cases.

Posting screenshots makes not much sense, because there is so little difference to see. :) So just try it. Needed are MaskTools and RemoveGrain.


function SPresso(clip clp, int "limit", int "bias", int "RGmode")
{
limit = default( limit, 2 )
bias = default( bias, 25 )
RGmode = default( RGmode, 4 )

LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)

expr = (limit<0) ? "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : "x y = x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"

yv12lutxy( clp, clp.removegrain(RGmode,-1), expr, U=2,V=2)
}
Note: Chroma is not touched at all. Spresso works only on the luma channel. Could be changed easily, however.

BTW: For those prefering tee over coffe, there's also a "TPresso" counterpart :D ... I just don't have it with me at the moment.

[Razorholt]

Thanks guys! I will try out your suggestions and post the results later on today. I've just finished the first test with a combinaision of:

Undot()
Unfilter(-5,-5)
BicubicResize()

Result: produced artifacts so I will give a 7 out of 10. (I will ellaborate more also later on)


Thanks,
- Dan

[Razorholt]

Ok, here is below the script I'm quite happy with. I still need to try blur() as suggested.

Anything I can add? :) Maybe a filter to sharpen the clip a little? Artifacts are still noticeable but not "annoying".



ConvertToYv12()
function SPresso(clip clp, int "limit", int "bias", int "RGmode")
{
limit = default( limit, 2 )
bias = default( bias, 25 )
RGmode = default( RGmode, 4 )

LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)

expr = (limit<0) ? "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : "x y = x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"

yv12lutxy( clp, clp.removegrain(RGmode,-1), expr, U=2,V=2)
}

Undot()
ConvertToYUY2()
Unfilter(-5,-5)
Temporalsoften(2,3,3,mode=2,scenechange=6)
FluxSmoothST(5,7)
crop(2,0,0,0)
GaussResize(852,480)

[DryFire]

You should probably make a seperate .avs or .avsi (auto loading) file for spresso. As is does that even call SPresso?

Also why the conversion to yuy2? IIRC unfilter, temporal soften and fluxsmooth all work in yv12.

[Razorholt]

ooh, and I have to test degrainmedian() too. What would be the recommended settings for that one?


Thanks,
- Dan

[Razorholt]

You should probably make a seperate .avs or .avsi (auto loading) file for spresso. As is does that even call SPresso?

Also why the conversion to yuy2? IIRC unfilter, temporal soften and fluxsmooth all work in yv12.


I tried to autoload SPresso.avs but it didn't work. The error message said that Avisynth couldn't load it.

[Tuesday]

Hey, I always use PixieDust to help with compressibilty in my rips, I know it produces subtle blocking artifacts, but i can never see them in the video when it's running and it more or less completly removes most noise.

'tis a little on the slow side though.

[Razorholt]

Hey, I always use PixieDust to help with compressibilty in my rips, I know it produces subtle blocking artifacts, but i can never see them in the video when it's running and it more or less completly removes most noise.

'tis a little on the slow side though.


Well, I wanted to try PixieDust but never found the right version I guess. Can you please advise me on this one?

Thanks,
- Dan

[Razorholt]

Another set of filters. This time I decided to rezise the clip from 852x480 to 800x450. That helped reducing the artifacts.


ConvertToYv12()
function SPresso(clip clp, int "limit", int "bias", int "RGmode")
{
limit = default( limit, 2 )
bias = default( bias, 25 )
RGmode = default( RGmode, 4 )

LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)

expr = (limit<0) ? "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : "x y = x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"

yv12lutxy( clp, clp.removegrain(RGmode,-1), expr, U=2,V=2)
}


Unfilter(-5,-5)
Temporalsoften(2,3,3,mode=2,scenechange=6)
FluxSmoothST(5,7)
blur(0.15)
BicubicResize(800,450)

[Didée]

In your first post, you talked about a "good quality source". Now you're always talking about artefacts that have to be reduced. Could you please decide what's the real deal?

The following script loads and uses Spresso correctly. Get the plugins, adapt the pathnames, and it'll work:

LoadPlugin("X:\path\to\MaskTools.dll (http://manao4.free.fr/MaskTools-v1.5.8.zip)")
LoadPlugin("X:\path\to\RemoveGrain.dll (http://home.arcor.de/kassandro/RemoveGrain/RemoveGrain.rar)")

Import("Y:\path\to\SPresso.avs")

XYZSource("Z:\path\to\YourSource")

SPresso()

The filtering chain you currently have gives a rather blurry output for sure. One should look into what the problem is with the actual source, and do specific filtering. Better then to produce this kind of universal mush. ;)

BTW, you'll have no fun with a framesize of 800*450. You should resize to 800*448.
(And in case you like mush, write "BicubicResize(800,448,1,0)" -- no more artefacts...) ;)


edit:

BTW2, I once put the "Dust" filter package online here (http://home.arcor.de/dhanselmann/_stuff/dust_v5.zip). Those who are afraid of Dust's potential blocking artefacts might want to use PixieDustPP (http://forum.doom9.org/showthread.php?p=727667#post727667) instead.

[Razorholt]

Ok Didée, here is the deal:

I have to compress a 25MB QuickTime file (movie trailer) down to 17MB (Mpeg-4) using MainConcept H.264. I wanted to keep the original screen resolution (854x480) but the quality wasn't that impressive so I decided to go with 800x450 (although I should go with 800x448 as per your suggestion :)).

I know that my script looks like a messy bunch of filters put together with the hope that the combinaison could get me to the result espected, but so far so good :)

Regarding PixieDustPP (who's dying to join the group, I know it for sure) I can't make it to work. After importing the AVS file and loading the 3 plugins required I get the following error:

Script error: There is no function named "pixiedust"
(C:\Program Files\AviSynth 2.5\plugins\PixieDustPP.avs, line 47)


Cheers.

[Razorholt]

I just got the PixieDustPP problem sorted :) I forgot to load LoadPluginEx, that's why it didn't work... I'm currently compressing the file wth that filter so I'll let you know the result.

It might be a great filter but the encoding time is a bit long. The time for the other tests varied between 41 and 52 minutes (VBR 2-pass), versus 1h42 with this pixiedustpp included. But we'll see...

- Dan

[Elic]

Mug Funky
> i've never seen undot produce an artefact...?
Undot() doesn't produce artifacts ( == additional details not exist in source video), just eliminates existing details with near-one-pixel size (especially fast-moving) and like "washes" fine-grained, slow-moving textures. Usually it is good, but sometimes it irritate like nothing else. :(

[esby]

BicubicResize()


I am missing something.
BicubicResize() versus what ?
If it is against lanczos resize, I'd say yes.
If it is against normal resize, I'd say no.
If it is reducting the image size, I'd say yes, but the bicubic is not important here.
If I had to use a resize for downsampling and helping compressibility I would probably use bilinear resizing.

esby

[Razorholt]

I think I have a great combinaison of filters here. Although I'm not sure about the order...



SPresso(2,33,4)
DeGrainMedian(limitY=5,limitUV=7,mode=1)
Undot()
Unfilter(-5,-5)
BilinearResize(800,448)

[Jeremy Duncan]

May I hop in with a small function? :)

SPresso() is one I use a lot (mostly for internal processing within more complex functions), but it also deals very well as a standalone.

(It doesn't matter if you like caffeine or not (I do) - SPresso stands for Spatial Pressdown.)

On my usual, usual-noisy DVB sources, compression gain usually is from 2% to 3% (light settings -> changes almost invisible) up to 10 to 12% (stronger settings -> slight, gentle softening, not very obvious).

Format:

SPresso( limit, bias, [RGmode] )

"limit" tells the maximum change to do to any pixel
"bias" is, well, something like "aggessivity": '20' is a very light setting, '33' is already quite strong.
"RGmode" tells the RemoveGrain mode to use. The default of "4" is the best in most cases.

Posting screenshots makes not much sense, because there is so little difference to see. :) So just try it. Needed are MaskTools and RemoveGrain.


function SPresso(clip clp, int "limit", int "bias", int "RGmode")
{
limit = default( limit, 2 )
bias = default( bias, 25 )
RGmode = default( RGmode, 4 )

LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)

expr = (limit<0) ? "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : "x y = x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"

yv12lutxy( clp, clp.removegrain(RGmode,-1), expr, U=2,V=2)
}
Note: Chroma is not touched at all. Spresso works only on the luma channel. Could be changed easily, however.

BTW: For those prefering tee over coffe, there's also a "TPresso" counterpart :D ... I just don't have it with me at the moment.

Could you please make a Spresso for Chroma.
Owen says that the Noise is mostly in Chroma, and I use HQDN3D(0.0, 2.0, 0.0, 4.0) so Chroma is affected but not Luma.
It would sure be nice if you could make a Spresso for Chroma only.

:script:

[Didée]

Supposely this "reduced-medianfilter" method isn't so awfully effective for removing chroma noise. Temporal filters should be a better choice here.

But, for what it's worth, here's SPresso extended to work on either luma, chroma, or both:

( EDIT 2006-09-04: corrected a BIG bug.)
function SPresso(clip clp, int "limit", int "bias", int "RGmode", int "limitC", int "biasC", int "RGmodeC")
{
limit = default( limit, 2 )
limitC = default( limitC, 4 )
bias = default( bias, 25 )
biasC = default( biasC, 50 )
RGmode = default( RGmode, 4 )
RGmodeC = default( RGmodeC, 0 )

yy = (RGmode ==0) ? 2 : 3
uv = (RGmodeC==0) ? 2 : 3
LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)
LIM1c = (limitC>0) ? string( round(limitC*100.0/biasC-1.0) ) : string( round(100.0/biasC) )
LIM2c = (limitC<0) ? "1" : string(limitC)
BIAc = string(biasC)

expr = (limit<0) ? "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : "x y - abs 0 <= x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"
exprC = (limit<0) ? "x y - abs "+LIM1c+" < x x 1 x y - x y - abs / * - ?"
\ : "x y - abs 0 <= x x "+LIM1c+" + y < x "+LIM2c+" + x "+LIM1c+" - y > x "+LIM2c+" - " \
+ "x 100 "+BIAc+" - * y "+BIAc+" * + 100 / ? ? ?"

# For (old) MaskTools v1.5.8 :
# yv12lutxy( clp, clp.removegrain(RGmode,RGmodeC), expr,exprC,exprC, Y=yy,U=uv,V=uv)

# For (new) MaskTools v2.x :
mt_lutxy( clp, clp.removegrain(RGmode,RGmodeC), yexpr=expr,uexpr=exprC,vexpr=exprC, Y=yy,U=uv,V=uv)

return( last )
}

Per default, chroma still is not processed, just passed through. To also process chroma, call SPresso() with "RGmodeC=4" (or 19, or 20). To not process luma, set "RGmode=0".

[frednerk33]

Sorry, Spresso is mentioned in a LimitedSharpen thread, http://forum.doom9.org/showthread.php?p=867137#post867137 but can some kind soul point me to an explanation of what Spresso does and when to apply it ? Jeremy's link http://forum.doom9.org/showthread.php?p=867137#post867137 has the some flash-looking code. ... but I'm unsure what is it that SPresso is meant to do and why. It seems to be used in the context of noise filtering ?

[Jeremy Duncan]

Thank you very much.

[n3w813]

So the new call with chorma processing only would be....

SPresso(0, 0, 0, 4, 50, 4)

Right?

[Jeremy Duncan]

So the new call with chorma processing only would be....

SPresso(0, 0, 0, 4, 50, 4)

Right?

The bolded parts are Chroma.

SPresso(0, 4, 0, 40, 1, 17)

[n3w813]

The bolded parts are Chroma.

SPresso(0, 4, 0, 40, 1, 17)

Are you sure...in the script it shows....

function SPresso(clip clp, int "limit", int "bias", int "RGmode", int "limitC", int "biasC", int "RGmodeC")

the variables with 'C' are the chroma settings right?

[Jeremy Duncan]

Are you sure...in the script it shows....

function SPresso(clip clp, int "limit", int "bias", int "RGmode", int "limitC", int "biasC", int "RGmodeC")

the variables with 'C' are the chroma settings right?

The Bolded are the Chroma.

limit = default( limit, 2 )
limitC = default( limitC, 4 )
bias = default( bias, 25 )
biasC = default( biasC, 50 )
RGmode = default( RGmode, 4 )
RGmodeC = default( RGmodeC, 0 )

SPresso(0, 4, 0, 40, 1, 17)

[foxyshadis]

NO NO NO, it's the order in the function definition that's important, and this is a good time to reiterate:

Don't call functions with bare arguments, use names, never worry about the order.

Otherwise it happens all the time, you upgrade a dll or script and the video turns green and looks vertically blurred, or whatever, because the order of arguments changed. Or an argument was swapped out with one with an entirely different meaning. Named optional arguments are one of the biggest conveniences of avisynth; even if the calls get wider they get much more understandable.

Btw, SPresso is essentially what SeeSaw does when you supply no denoised argument, right? (Its effect is that high freqency noise is smoothed out somewhat, though I'm not entirely sure of the details; a quick n' dirty one.)

[n3w813]

Don't call functions with bare arguments, use names, never worry about the order.



So.....if using MT......
MT("SPresso(limit=0, bias=0, RGmode=0, limitC=4, biasC=50, RGmodeC=4)",3)

Sorry, I'm a REAL n3w813 when it comes to avisynth scripts. :P

[foxyshadis]

Either order will work just fine now; the precise values are open to taste. :p

[Jeremy Duncan]

So.....if using MT......
MT("SPresso(limit=0, bias=0, RGmode=0, limitC=4, biasC=50, RGmodeC=4)",3)

Sorry, I'm a REAL n3w813 when it comes to avisynth scripts. :P

MT("SPresso(limit=0, limitC=4, bias=0, biasC=40, RGmode=1, RGmodeC=17)",3)

The colors will be messed up if you don't use at least #1 on RGmode.
I've bolded in the quote the setting that needs to be at least # 1.

[Boulder]

RGMode=0 should mean that luma is simply copied as it is - it shouldn't do anything to the colors. If it does, there's something wrong in Didée's function.

[Jeremy Duncan]

RGMode=0 should mean that luma is simply copied as it is - it shouldn't do anything to the colors. If it does, there's something wrong in Didée's function.

Using this code;
SPresso(limit=0, limitC=4, bias=0, biasC=40, RGmode=1, RGmodeC=17)

And setting RGmode=0, the colors go way off. It's very ugly.

[Didée]

I can't reproduce that. There's nothing wrong in the function, as far as I can see, and RGmode=0 / limit=0 works fine for me.


@ foxyshadis

Lol, good description of might happen. You forgot "video might start playing backwards".

Btw, SPresso is essentially what SeeSaw does when you supply no denoised argument, right? (Its effect is that high freqency noise is smoothed out somewhat, though I'm not entirely sure of the details; a quick n' dirty one.)
Yes, it's the same principle, only that SPresso's defaults are weaker. In SeeSaw, it's stronger because of the "indirect processing" that is done ... the denoising can be stronger, because more denoising only means there will be less detail (and thereby less artefacts) that will be sharpened.

Generally, Spresso is just a limited Removegrain(4). Put in words, SPresso(RGmode=4,bias=25,limit=2) reads:

"Apply 25% of RemoveGrain(4), with a maximum pixelchange of +/- 2".

[BigDid]

... Generally, Spresso is just a limited Removegrain(4). Put in words, SPresso(RGmode=4,bias=25,limit=2) reads:

"Apply 25% of RemoveGrain(4), with a maximum pixelchange of +/- 2".
Thanks, crystal clear.

I really need to learn scripting more :o

Did

[frednerk33]

Yes, it's the same principle, only that SPresso's defaults are weaker. In SeeSaw, it's stronger because of the "indirect processing" that is done ... the denoising can be stronger, because more denoising only means there will be less detail (and thereby less artefacts) that will be sharpened.

Generally, Spresso is just a limited Removegrain(4). Put in words, SPresso(RGmode=4,bias=25,limit=2) reads:

"Apply 25% of RemoveGrain(4), with a maximum pixelchange of +/- 2".
Teriffic, thanks for the clarfication.

[Jeremy Duncan]

I just thought I'd point out that this spresso suffers the same problem as the old deblock_qed, with a couple of incorrect = signs in the lut expr. (Should be ==.)

Is this the correct code ?

function SPresso(clip clp, int "limit", int "bias", int "RGmode", int "limitC", int "biasC", int "RGmodeC")
{
limit = default( limit, 2 )
limitC = default( limitC, 4 )
bias = default( bias, 25 )
biasC = default( biasC, 50 )
RGmode = default( RGmode, 4 )
RGmodeC = default( RGmodeC, 0 )

yy = (RGmode ==0) ? 2 : 3
uv = (RGmodeC==0) ? 2 : 3
LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)
LIM1c = (limitC>0) ? string( round(limitC*100.0/biasC-1.0) ) : string( round(100.0/biasC) )
LIM2c = (limitC<0) ? "1" : string(limitC)
BIAc = string(biasC)

expr = (limit<0) ? "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : "x y == x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"
exprC = (limit<0) ? "x y - abs "+LIM1c+" < x x 1 x y - x y - abs / * - ?"
\ : "x y == x x "+LIM1c+" + y < x "+LIM2c+" + x "+LIM1c+" - y > x "+LIM2c+" - " \
+ "x 100 "+BIAc+" - * y "+BIAc+" * + 100 / ? ? ?"

# For (old) MaskTools v1.5.8 :
# yv12lutxy( clp, clp.removegrain(RGmode,RGmodeC), expr,exprC,exprC, Y=yy,U=uv,V=uv)

# For (new) MaskTools v2.x :
mt_lutxy( clp, clp.removegrain(RGmode,RGmodeC), yexpr=expr,uexpr=exprC,vexpr=exprC, Y=yy,U=uv,V=uv)

return( last )
}

[Didée]

Not bad. But I prefer the one some posts above (http://forum.doom9.org/showthread.php?p=867316#post867316). :)

[boombastic]

i get this error using spresso:

TemporalSoften::YUV2 source must be multiple of 4

the script is:

SetMtMode(2)
mpeg2source("F:\1x02\VIDEO_TS\VTS_01_1.d2v")
import("d:\Programmi\AviSynth 2.5\plugins\SeeSaw.avs")
import("d:\Programmi\AviSynth 2.5\plugins\mvbob.avs")
import("d:\Programmi\AviSynth 2.5\plugins\spresso.avs")
ColorMatrix(d2v="F:\1x02\VIDEO_TS\VTS_01_1.d2v")
Crop(8,80,-32,-80)
SeparateFields()
a=SelectEven().SPresso(limit=8, limitC=8, bias=100, biasC=100, RGmode=1, RGmodeC=17).SeeSaw(NRlimit=2, NRlimit2=2, Sstr=1.0, Slimit=7, Spower=2, Sdamplo=8, Szp=10, bias=1)
b=SelectOdd().SPresso(limit=8, limitC=8, bias=100, biasC=100, RGmode=1, RGmodeC=17).SeeSaw(NRlimit=2, NRlimit2=2, Sstr=1.0, Slimit=7, Spower=2, Sdamplo=8, Szp=10, bias=1)
weave()
#TomsMoComp(1,3,1)
mvbob()
LumaYV12(0,0.9)

[kiwisrgreen]

I'm doing the 2-pass encode thing (anime) and the first pass was 307MB (without audio)
70% of that is 214.9MB
audio is 29.1 MBMB
so second pass should 244mb or so...
so I did that but it looked like it needs to be a little bigger like the original size?

filters used were:
ConvertToYV12()
ColorYUV(gain_u=9)
Tweak(cont=.98,bright=3)
Undot()
hqdn3d()
removegrain()
warpsharp(depth=30,blur=3,bump=65)

original anime raw (encoded by xvid already) is 250MB, fairly clean there is some noticeable noise during certain scenes, and sometimes it looks like compression noise.

the encoded video looks better than original but size is pretty much the same.

[Jeremy Duncan]

Here's a Spresso code you can try.

SPresso(limit=2, limitC=13, bias=100, biasC=100, RGmode=17, RGmodeC=17)

You need the masktools.dll from this link.
masktools_alpha (2.0a30). (http://manao4.free.fr/masktools-v2.0a30.zip)

And removegrain.dll from this link.
RemoveGrain 1.0 Prerelease (http://home.arcor.de/kassandro/RemoveGrain/RemoveGrain.rar)

[Jeremy Duncan]

Didée,

The Chroma setting doesn't work.
The Luma setting works fine.

Setting the Luma Limit to 100 effectively blurs the picture, but setting the Chroma Limit to 100 does nothing.
I've run other tests that showed the Spresso is entirely dependant on the Luma Limit, Bias, and RGmode.

Here's the setting I made.
SPresso(limit=2, limitC=13, bias=100, biasC=100, RGmode=17, RGmodeC=17)

Also setting the Luma RGmode to 1 disables it, leaving a non-functional Spresso.

[Didée]

All of that is plain wrong. :)

Dunno what you did - perhaps you imported the old version instead of the new one ... ?

Loading some random content, and applying

Spresso(RGmode=1,RGmodeC=17,bias=100,biasC=100,limit=16,limitC=16)

the achieved effect was

http://img253.imageshack.us/img253/1308/spressouvdifferencewo3.th.png (http://img253.imageshack.us/my.php?image=spressouvdifferencewo3.png)


In respect to "RGmode=1 does nothing" and "Chroma Modes do nothing", I'm seeing pretty much differences there .... so: everything works exactly as it should. :)

Fact is, that spatial denoising often isn't all that awfully effective for chroma noise. That's what I told you formerly, and that's why the initial SPresso script worked only on luma plane. But you cried for a chroma mode, and so you got it. :)

[Jeremy Duncan]

everything works exactly as it should. :)

Fact is, that spatial denoising often isn't all that awfully effective for chroma noise. That's what I told you formerly, and that's why the initial SPresso script worked only on luma plane. But you cried for a chroma mode, and so you got it. :)

Interesting.

Thanks for clearing that up Didée.

[koolcracker]

Do you actually have a TPresso? because that would be very helpful. Well, assuming that T stood for Temporal.

[Didée]

*Issue solved, post deleted*

[Jeremy Duncan]

Here's the Spresso setting I made.

SPresso(limit=2, limitC=2, bias=76, biasC=76, RGmode=4, RGmodeC=4)

I'm sorry for the mistake Didée.
Thank you for your time and patience, I appreciate all you've done.

[koolcracker]

So is that a no for a Temporal Pressdown?

[Didée]

Doing a temporal counterpart is very easy. Basically, just that "RemoveGrain(..)" has to be replaced with a temporal filter. That's all.

And that's where the questioning begins: what temporal filter to use? Clense? FluxSmoothT? TemporalSoften? TTempSmooth? MedianBlurT? There are so many ...
Also, the weighting/limiting could be extended, like always using a lower minimum plus some weighting on top of that ...

Many ways to blow up the function to fill three or four pages ...
... for something where the actual filtering is just one single line. ;)

[foxyshadis]

My first thought was
Soothe(last,TemporalSoften(...),40)
or perhaps vice versa... but who knows, that might not work at all. ;p

[Didée]

Well, you can do that ... like you can put mustard onto a cream cake. :)

Soothe reverts the work of [filter] in those places where the difference achieved by [filter] is temporally unstable.
Now, the difference achieved by a temporal filter is supposed to be temporally unstable, so ... ;)

Soothe's corresponding counterpart for temporal filters would be done through spatial processing. (Like demonstrated e.g. in SafeTemporalSoften (http://forum.doom9.org/showthread.php?p=875442#post875442))

[Jeremy Duncan]

Didée,

If you make a Tpresso, please include Chroma denoising, and since it would be used in realtime it would be nice if it was fast.

Thanks for your time in reading and considering this.

:)

[J-Wo]

Didee, just discovered this function after your mentioned of it to me in another thread. Great work! I tried it out at the default settings (limit=2, bias=25, rgmode=4) and the results were very promising. I did an OPV encode with 3 different bias settings and compared their compressibility to one without spresso. I also threw into the mix one that used removegrain(mode=5) instead of spresso. Here's what I found:

removegrain 5 --> 5.35% compression
bias 25 --> 4.82%
bias 26 --> 7.42%
bias 33 --> 8.12%

I then did a 2 pass encode with these same filters to see how much detail was preserved by the various filters. On my 42" LCD rear projection HDTV they all looked great! But to my surprise, I think the higher bias looked the best. I think that was because the higher compression allowed the encoder to do it's... you know, encoding magic better. The result wasn't too soft at all (your original post led me to believe that a bias=33 was a very strong setting).

P.S. I believe someone mentioned it before, but the best way to use this function is to save it in your Avisynth plugin file as SPresso.avsi. This way you never have to load the function with the Import() line in your scripts.

P.P.S. You should rename this thread to put Spresso in the title! It'll make searching for help easier

[foxyshadis]

But if he did that, people might actually find and start using his filters, and ask for help. So he just sneaks them into someone else's threads. ;)

This is one of the few I can't seem to get much of a handle on, probably because it's hard to see the subtle differences that make a big difference to the encoder.

[J-Wo]

Well for me I do a lot of DVD9->DVD5 compression using DVD-RB. And while most of my sources are clean film material, a little grain removal (that is barely detectable to the naked eye) will help with compression. So far I think this filter is great!

[AGKnotUser]

Well for me I do a lot of DVD9->DVD5 compression using DVD-RB. And while most of my sources are clean film material, a little grain removal (that is barely detectable to the naked eye) will help with compression. So far I think this filter is great!

Could you post your settings? I'm also using DVD-RB and want to use this filter for the same purpose.

[Jeremy Duncan]

Here's what I use;

Spresso(limit=8, limitC=4, bias=59, biasc=100, rgmode=17, rgmodec=17)

[AGKnotUser]

Thank you for your suggestion.

[Vesi]

Hi Didee.
if i use SPresso(RGmode=4,bias=25,limit=2) on source which is very good source and doesn't need strong denoising + sharping what you thing? or any other suggestion?

[real.finder]

# SPresso by Didée, r.f mod in 2020.04.23 for HBD
function SPresso(clip clp, float "limit", float "bias", int "RGmode", float "limitC", float "biasC", int "RGmodeC")
{
sisavs26 = !(VersionNumber() < 2.60)
limit = default( limit, 2 )
limitC = default( limitC, 4 )
bias = default( bias, 25 )
biasC = default( biasC, 50 )
RGmode = default( RGmode, 4 )
RGmodeC = default( RGmodeC, 0 )

yy = (RGmode ==0) ? 2 : 3
uv = (RGmodeC==0) ? 2 : 3
LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)
LIM1c = (limitC>0) ? string( round(limitC*100.0/biasC-1.0) ) : string( round(100.0/biasC) )
LIM2c = (limitC<0) ? "1" : string(limitC)
BIAc = string(biasC)

expr = (limit<0) ? sisavs26 ? "x y - abs "+LIM1+" scalef < x x 1 x y - x y - abs / * - ?" : "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : sisavs26 ? "x y - abs 0 <= x x "+LIM1+" scalef + y < x "+LIM2+" scalef + x "+LIM1+" scalef - y > x "+LIM2+" scalef - " \
+ "x 100 scalef "+BIA+" scalef - * y "+BIA+" scalef * + 100 scalef / ? ? ?" \
: "x y - abs 0 <= x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"
exprC = (limit<0) ? sisavs26 ? "x y - abs "+LIM1c+" scalef < x x 1 x y - x y - abs / * - ?" : "x y - abs "+LIM1c+" < x x 1 x y - x y - abs / * - ?"
\ : sisavs26 ? "x y - abs 0 <= x x "+LIM1c+" scalef + y < x "+LIM2c+" scalef + x "+LIM1c+" scalef - y > x "+LIM2c+" scalef - " \
+ "x 100 scalef "+BIAc+" scalef - * y "+BIAc+" scalef * + 100 scalef / ? ? ?" \
: "x y - abs 0 <= x x "+LIM1c+" + y < x "+LIM2c+" + x "+LIM1c+" - y > x "+LIM2c+" - " \
+ "x 100 "+BIAc+" - * y "+BIAc+" * + 100 / ? ? ?"

# For (old) MaskTools v1.5.8 :
# yv12lutxy( clp, clp.removegrain(RGmode,RGmodeC), expr,exprC,exprC, Y=yy,U=uv,V=uv)

# For (new) MaskTools v2.x :
sisavs26 ? mt_lutxy( clp, clp.removegrain(RGmode,RGmodeC), yexpr=expr,uexpr=exprC,vexpr=exprC, scale_inputs=limit<0 ? "floatf" : "none", use_expr=1, Y=yy,U=uv,V=uv) : mt_lutxy( clp, clp.removegrain(RGmode,RGmodeC), yexpr=expr,uexpr=exprC,vexpr=exprC, Y=yy,U=uv,V=uv)

return( last )
}

again, I want volunteers for testing with HBD :p

[real.finder]

another update, should work with 1080 and up

# SPresso by Didée, r.f mod in 2020.05.07 for HBD
function SPresso(clip clp, float "limit", float "bias", int "RGmode", float "limitC", float "biasC", int "RGmodeC", bool "HD", int "radius")
{
sisavs26 = !(VersionNumber() < 2.60)
limit = default( limit, 2 )
limitC = default( limitC, 4 )
bias = default( bias, 25 )
biasC = default( biasC, 50 )
RGmode = default( RGmode, 4 )
RGmodeC = default( RGmodeC, 0 )
h = clp.height()
ishd = h > 720
HD = default( HD, ishd )
radius = default( radius, round(h/540.0) )

yy = (RGmode ==0) ? 2 : 3
uv = (RGmodeC==0) ? 2 : 3
LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)
LIM1c = (limitC>0) ? string( round(limitC*100.0/biasC-1.0) ) : string( round(100.0/biasC) )
LIM2c = (limitC<0) ? "1" : string(limitC)
BIAc = string(biasC)

expr = (limit<0) ? sisavs26 ? "x y - abs "+LIM1+" scalef < x x 1 x y - x y - abs / * - ?" : "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : sisavs26 ? "x y - abs 0 <= x x "+LIM1+" scalef + y < x "+LIM2+" scalef + x "+LIM1+" scalef - y > x "+LIM2+" scalef - " \
+ "x 100 scalef "+BIA+" scalef - * y "+BIA+" scalef * + 100 scalef / ? ? ?" \
: "x y - abs 0 <= x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"
exprC = (limit<0) ? sisavs26 ? "x y - abs "+LIM1c+" scalef < x x 1 x y - x y - abs / * - ?" : "x y - abs "+LIM1c+" < x x 1 x y - x y - abs / * - ?"
\ : sisavs26 ? "x y - abs 0 <= x x "+LIM1c+" scalef + y < x "+LIM2c+" scalef + x "+LIM1c+" scalef - y > x "+LIM2c+" scalef - " \
+ "x 100 scalef "+BIAc+" scalef - * y "+BIAc+" scalef * + 100 scalef / ? ? ?" \
: "x y - abs 0 <= x x "+LIM1c+" + y < x "+LIM2c+" + x "+LIM1c+" - y > x "+LIM2c+" - " \
+ "x 100 "+BIAc+" - * y "+BIAc+" * + 100 / ? ? ?"

# For (old) MaskTools v1.5.8 :
# yv12lutxy( clp, clp.removegrain(RGmode,RGmodeC), expr,exprC,exprC, Y=yy,U=uv,V=uv)

# For (new) MaskTools v2.x :
sisavs26 ? mt_lutxy( clp, HD ? clp.minblur(radius,uv=uv) : clp.removegrain(RGmode,RGmodeC), yexpr=expr,uexpr=exprC,vexpr=exprC, scale_inputs=limit<0 ? "floatf" : "none", use_expr=1, Y=yy,U=uv,V=uv) : mt_lutxy( clp, HD ? clp.minblur(radius,uv=uv) : clp.removegrain(RGmode,RGmodeC), yexpr=expr,uexpr=exprC,vexpr=exprC, Y=yy,U=uv,V=uv)

return( last )
}