Using Avisynth to help compressing files

[Razorholt]

I was wondering, which filters may be used to help compressing a good quality file (not anime)?

undot()
Fluxsmooth()

Good start?



Thanks,
- Dan

[sh0dan]

Probably the best filter: BicubicResize() to a smaller size.

Otherwise have a look at this thread.

[Razorholt]

Thanks shOdan.

- Dan

[Elic]

Razorholt
I shoudn't advise to use Undot() for improving compressability of hi-quality video because it eliminates small fast-moving details (like rain-drops or snow-flakes) almost completely. Instead, I prefer to use light temporal denoising like

Temporalsoften(2,3,3,mode=2,scenechange=6)
FluxSmoothST(5,7)

(I stealed this from scripts generated by GordianKnot).

Also, light blur() (with strength of 0.15 or so) before BicubicResize() might gain the compressability, but it highly depends on source video.

[Mug Funky]

i've never seen undot produce an artefact...?

removedirt can be a bit agressive as far as removing objects rather than just noise, but for compressibility gain versus detail it can't be beaten. if used with motion compensation (especially the new stuff being developed lately) it can give a practically artefact-free compressibility boost.

degrainmedian is also very good - doesn't give blocks like removedirt can if not used judiciously, and still gives a handy compressibility gain.

[IanB]

Perhaps the new GaussResize(), adjust P to select the right amount of blur

[Didée]

May I hop in with a small function?

SPresso() is one I use a lot (mostly for internal processing within more complex functions), but it also deals very well as a standalone.

(It doesn't matter if you like caffeine or not (I do) - SPresso stands for Spatial Pressdown.)

On my usual, usual-noisy DVB sources, compression gain usually is from 2% to 3% (light settings -> changes almost invisible) up to 10 to 12% (stronger settings -> slight, gentle softening, not very obvious).

Format:

SPresso( limit, bias, [RGmode] )

"limit" tells the maximum change to do to any pixel
"bias" is, well, something like "aggessivity": '20' is a very light setting, '33' is already quite strong.
"RGmode" tells the RemoveGrain mode to use. The default of "4" is the best in most cases.

Posting screenshots makes not much sense, because there is so little difference to see. So just try it. Needed are MaskTools and RemoveGrain.

function SPresso(clip clp, int "limit", int "bias", int "RGmode")
{
limit  = default( limit,  2 )
bias   = default( bias,  25 )
RGmode = default( RGmode, 4 )

LIM1  = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2  = (limit<0) ? "1" : string(limit)
BIA   = string(bias)

expr  = (limit<0) ?   "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
 \                :   "x y = x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \ 
                    + "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"

yv12lutxy( clp, clp.removegrain(RGmode,-1), expr, U=2,V=2)
}

Note: Chroma is not touched at all. Spresso works only on the luma channel. Could be changed easily, however.

BTW: For those prefering tee over coffe, there's also a "TPresso" counterpart ... I just don't have it with me at the moment.

[Razorholt]

Thanks guys! I will try out your suggestions and post the results later on today. I've just finished the first test with a combinaision of:

Undot()
Unfilter(-5,-5)
BicubicResize()

Result: produced artifacts so I will give a 7 out of 10. (I will ellaborate more also later on)


Thanks,
- Dan

[Razorholt]

Ok, here is below the script I'm quite happy with. I still need to try blur() as suggested.

Anything I can add? Maybe a filter to sharpen the clip a little? Artifacts are still noticeable but not "annoying".

ConvertToYv12()
function SPresso(clip clp, int "limit", int "bias", int "RGmode")
{
limit = default( limit, 2 )
bias = default( bias, 25 )
RGmode = default( RGmode, 4 )

LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)

expr = (limit<0) ? "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : "x y = x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"

yv12lutxy( clp, clp.removegrain(RGmode,-1), expr, U=2,V=2)
}

Undot()
ConvertToYUY2()
Unfilter(-5,-5)
Temporalsoften(2,3,3,mode=2,scenechange=6)
FluxSmoothST(5,7)
crop(2,0,0,0)
GaussResize(852,480)

[DryFire]

You should probably make a seperate .avs or .avsi (auto loading) file for spresso. As is does that even call SPresso?

Also why the conversion to yuy2? IIRC unfilter, temporal soften and fluxsmooth all work in yv12.

[Razorholt]

ooh, and I have to test degrainmedian() too. What would be the recommended settings for that one?


Thanks,
- Dan

[Razorholt]

Originally Posted by DryFire

You should probably make a seperate .avs or .avsi (auto loading) file for spresso. As is does that even call SPresso?

Also why the conversion to yuy2? IIRC unfilter, temporal soften and fluxsmooth all work in yv12.

I tried to autoload SPresso.avs but it didn't work. The error message said that Avisynth couldn't load it.

[Tuesday]

Hey, I always use PixieDust to help with compressibilty in my rips, I know it produces subtle blocking artifacts, but i can never see them in the video when it's running and it more or less completly removes most noise.

'tis a little on the slow side though.

[Razorholt]

Originally Posted by Tuesday

Hey, I always use PixieDust to help with compressibilty in my rips, I know it produces subtle blocking artifacts, but i can never see them in the video when it's running and it more or less completly removes most noise.

'tis a little on the slow side though.

Well, I wanted to try PixieDust but never found the right version I guess. Can you please advise me on this one?

Thanks,
- Dan

[Razorholt]

Another set of filters. This time I decided to rezise the clip from 852x480 to 800x450. That helped reducing the artifacts.

ConvertToYv12()
function SPresso(clip clp, int "limit", int "bias", int "RGmode")
{
limit = default( limit, 2 )
bias = default( bias, 25 )
RGmode = default( RGmode, 4 )

LIM1 = (limit>0) ? string( round(limit*100.0/bias-1.0) ) : string( round(100.0/bias) )
LIM2 = (limit<0) ? "1" : string(limit)
BIA = string(bias)

expr = (limit<0) ? "x y - abs "+LIM1+" < x x 1 x y - x y - abs / * - ?"
\ : "x y = x x "+LIM1+" + y < x "+LIM2+" + x "+LIM1+" - y > x "+LIM2+" - " \
+ "x 100 "+BIA+" - * y "+BIA+" * + 100 / ? ? ?"

yv12lutxy( clp, clp.removegrain(RGmode,-1), expr, U=2,V=2)
}


Unfilter(-5,-5)
Temporalsoften(2,3,3,mode=2,scenechange=6)
FluxSmoothST(5,7)
blur(0.15)
BicubicResize(800,450)

[Didée]

In your first post, you talked about a "good quality source". Now you're always talking about artefacts that have to be reduced. Could you please decide what's the real deal?

The following script loads and uses Spresso correctly. Get the plugins, adapt the pathnames, and it'll work:

LoadPlugin("X:\path\to\MaskTools.dll")
LoadPlugin("X:\path\to\RemoveGrain.dll")

Import("Y:\path\to\SPresso.avs")

XYZSource("Z:\path\to\YourSource")

SPresso()

The filtering chain you currently have gives a rather blurry output for sure. One should look into what the problem is with the actual source, and do specific filtering. Better then to produce this kind of universal mush.

BTW, you'll have no fun with a framesize of 800*450. You should resize to 800*448.
(And in case you like mush, write "BicubicResize(800,448,1,0)" -- no more artefacts...)


edit:

BTW2, I once put the "Dust" filter package online here. Those who are afraid of Dust's potential blocking artefacts might want to use PixieDustPP instead.

[Razorholt]

Ok Didée, here is the deal:

I have to compress a 25MB QuickTime file (movie trailer) down to 17MB (Mpeg-4) using MainConcept H.264. I wanted to keep the original screen resolution (854x480) but the quality wasn't that impressive so I decided to go with 800x450 (although I should go with 800x448 as per your suggestion ).

I know that my script looks like a messy bunch of filters put together with the hope that the combinaison could get me to the result espected, but so far so good

Regarding PixieDustPP (who's dying to join the group, I know it for sure) I can't make it to work. After importing the AVS file and loading the 3 plugins required I get the following error:

Script error: There is no function named "pixiedust"
(C:\Program Files\AviSynth 2.5\plugins\PixieDustPP.avs, line 47)


Cheers.

[Razorholt]

I just got the PixieDustPP problem sorted I forgot to load LoadPluginEx, that's why it didn't work... I'm currently compressing the file wth that filter so I'll let you know the result.

It might be a great filter but the encoding time is a bit long. The time for the other tests varied between 41 and 52 minutes (VBR 2-pass), versus 1h42 with this pixiedustpp included. But we'll see...

- Dan

[Elic]

Mug Funky
> i've never seen undot produce an artefact...?
Undot() doesn't produce artifacts ( == additional details not exist in source video), just eliminates existing details with near-one-pixel size (especially fast-moving) and like "washes" fine-grained, slow-moving textures. Usually it is good, but sometimes it irritate like nothing else.

[esby]

BicubicResize()

I am missing something.
BicubicResize() versus what ?
If it is against lanczos resize, I'd say yes.
If it is against normal resize, I'd say no.
If it is reducting the image size, I'd say yes, but the bicubic is not important here.
If I had to use a resize for downsampling and helping compressibility I would probably use bilinear resizing.

esby