help to increase compressibility a very noisy source

[phate89]

Hi. I'm trying to rip a bluray (without resizing it) that is very difficult to compress.
I usually don't use a lot of avisynth filters, usually i use ffvideosource as input (i remux the bd to mkv to be able to use fffvideosource with make mkv) and smdegrain with very low settings (tr=1, prefilter=0, sometimes contrasharp=true) so i'm not an expert and i don't know what to use in this difficult rip. I tried some filter but the result was overfiltered (removed even too much grain and washed out all the detail) or with not so much gain.
Here's a sample of the bd (i took a very noiisy part, cut with mkvmerge from my remux):

http://www.sendspace.com/file/b8brqb

Any kind of help is really appreciated

[Selur]

Have you tried a simple 'TemporalDegrain()' ? (http://avisynth.org/mediawiki/Temporal_Degrain)

[phate89]

Originally Posted by Selur

Have you tried a simple 'TemporalDegrain()' ? (http://avisynth.org/mediawiki/Temporal_Degrain)

i'm trying now with default settings, it actually removes very well the grain but doing that it also took a lot of detail.
I don't think it's possible to preserve it anyways...
The compressibility gain is awesome but i don't know if it is worth it..
I will try to reduce the strength..

[LemMotlow]

xxxx6

[phate89]

Originally Posted by LemMotlow

This video looks horrible. It's dark as hell. Dark detail and even some lower midtones have been crushed beyond repair. You'll get better advice if you submit a piece of the original. If this sample is the unprocessed original, grain is a long way from the vid's worst problems.

this piece is unprocessed (as you can see h264, not x264 and the bitrate is high).

The grain is my priority because if i try to rip it with a decent quality i get almost the original file size, and rip to get the same size it's pointless..

[LemMotlow]

xxxx7

[phate89]

Originally Posted by LemMotlow

The original shows only the natural fine-grain of movie film, which was how it was created. Kill the film grain, kill the detail. That's the least of the video's problems. You're trying to filter seriously bad black levels. No wonder you get artifacts and no detail.

ED: If you use SmoothLevels or lower contrast to bring up the black levels, then work with a Rec709 HD color matrix, you'll see that the "grain" is chroma noise. I'd go for chroma cleaners (FFT3DFilter?) rather than hitting any excess grain first. The original looks as if darks are badly crushed to begin with, but you can probably save some dark detail by fixing levels first.

I'd suggest FFT3DFilter followed by TemporalDegrain, both at defaults. This will leave some film grain, which means you'll retain the detail that's "inside" the film grain. If you try to totally smooth the video to look like DV, it will appear plastic.

I didn't try yet to work on colors so i'm a newbie.
About color matrix, shouldn't be already rec709 from bd h264 sources?
And how should i use smoothlevels (or smoothadjust?) to get lower contrast?
I tried with only temporaldegrain that already prefilter with fft3dfilter and with default settings (fft3dfilter sigma=16 and mvdegrain 2) it cleans everything with a very washed out result...

[Didée]

I don't see why LemMotlow belabours chroma noise so much? Chroma noise is a rather minor problem here. Also, yes, the shadow parts are somewhat crushed - however there is not awfully much that could be "fixed". Dark areas without detail are dark areas without detail. Unless they are turned into brightened areas without detail. But then - of course, when trying to force some "very strong" levels manipulation on the near-black parts, it is quite possible that thereof some chroma problems come into view.

Regarding compressibility, it's surely the luma noise grain that hurts. x264/CRF18/slow/film comes out at 36000 kbit/s on this sample, which is not little.

Would you consider stepping down to 720p? Me, I surely would! The NR-processing required for a compressible result with good detail & sharpness will be VERY slow @ 1080p. Excessive MVTools-usage on FullHD is no fun. 720p is so much easier ... and in case playback is done via PC again, Upsize+FineSharp during playback makes things look quite similar to 1080p, again.

_____

BTW, especially with "seasonal series" from the U.S., I have often the impression that some noticeable black-crush is going on. I dunno, but *perhaps* it is related to gamma handling during the mastering chain? In Rec.709, gamma is not a purely Gamma 2.2, but a modification with a linear ramp in the near-black area:


(blue = native gamma 2.2, green = Rec.709 Gamma definition)

If recording is done "blue", but during processing it is assumed to have-been "green" (or some similar mishandling during processing), this could explain the black crush that is often to be seen.

[phate89]

Encoding time it's not a big deal for me (good pc, time to waste), i usually encode in placebo/subme 10/me_range 32 so i'm used to slow encodings.
Refining during play it is because for now i'm stuck with a cheap htpc.
With other tv shows should be ok to simply switch to 720p, but with this i'd prefer to keep it at 1080p... If i can't a good compression improvement i will probably keep the dumped mkv...

About the crushed black i saw it in other shows, but not all luckily. I don't understand why tv shows have to get a worst handling than movies (with some exception of course). I tried to take once the itunes version and it is even worse, the colours are completely different, all greenish...

[Didée]

OK, then today evening I'll revisit the script I started to fiddle with yesterday. Who knows, perhaps ...

Encoding time it's not a big deal for me (good pc, time to waste)

... perhaps I'ill manage to change your mind. When speed drops into the low tenths-of-fps range, the "speed-is-no-issue" voices often get quiet.

BTW, when answering to the post directly above, there's no point in placing a full-quote.

[phate89]

Originally Posted by Didée

OK, then today evening I'll revisit the script I started to fiddle with yesterday. Who knows, perhaps ...

Thanks a lot for your help... I'm keeping my expectations low because it's probably impossible to do but if someone can do it it's you

Originally Posted by Didée

... perhaps I'ill manage to change your mind. When speed drops into the low tenths-of-fps range, the "speed-is-no-issue" voices often get quiet.

Try me, i'm ready.

[LemMotlow]

xxxx5

[Didée]

@phate89 - I'm not really sure what you expect or how you judge denoising results. So, please rate the following result:

To you, is the following a "good" result, or is it more like "no thanks, too much detail loss" :
Slow-Denoise (24 MB)

Technically: with x264 --preset slow --tune film --CRF 18 :

no filtering: 35976 kbit/s (6.30 fps)
slow filtering: 11130 kbit/s (0.44 fps)

I.e. 69% reduction of bitrate, but 14 times longer to process/encode.

[phate89]

Originally Posted by Didée

@phate89 - I'm not really sure what you expect or how you judge denoising results. So, please rate the following result:

To you, is the following a "good" result, or is it more like "no thanks, too much detail loss" :
Slow-Denoise (24 MB)

Technically: with x264 --preset slow --tune film --CRF 18 :

no filtering: 35976 kbit/s (6.30 fps)
slow filtering: 11130 kbit/s (0.44 fps)

I.e. 69% reduction of bitrate, but 14 times longer to process/encode.

To start, this is the best result in retaining detail i saw. Even worse cleaning solutions took away more detail than your solution.
The compression gain is even more than expected and the temporal noise (is this the definition of that stuff?) is cleaned very well.
I tried to play it and in that there's something a little bit strange on the sy, the static ponts (due to the high denoising).. It is because i know it was terrible so i look at it, without knowing that i'd never notice that.
After i watched the original and was more unnatural than your rip, so i want definnitely to give it a chance at least with a 5% to see how it works with other parts of the video.
Maybe leaving or putting back a little bit of noise will save from that little strange behaviour?

[Didée]

Yeah, that sort of "static grain" is a very common issue when the source starts with really strong grain, and the main denoising process is temporal filtering. It's hard to come by, unless some sort of spatial denoising/smoothing is thrown into the chain. But with spatial denoising in the game, it is very likely to cause blurring real detail where you don't want to have it.

Another approach:

-edit-
(Buggy script) -- Split_slowdenoise-3c_CRF18.mkv (31 MB)
(Correct script) -- Split_slowdenoise-3d_CRF18.mkv (29 MB)

This issues mainly temporal filtering, with additional spatial pre-filtering, masked to target by brightness/chroma masking.
Compressibility is somewhat lower than before (now 14500 13570 kbit/s @ CRF18/slow/film, compared to 36000 without filtering). Detail retention is different than before - a little less here, a little more there. In overall balance, I think it is better than before.

The used script. It is very slow, and it's possible that you cannot use it in a direct script-->x264 chain when using x264 with "exhaustive" settings (in particular high mbtree lookahead).

edit: corrected vector bug (delta value)

SetMemoryMax(1000)

# LoadPlugin( ...
#
#   MVTools2.dll
#   mt_masktools-25.dll
#   RemoveGrain.dll
#   Repair.dll
#   MedianBlur.dll



pel1 = 1   # you can use pel=2, 
pel2 = 1   # but with pel=1 the script is already slow & memory-hungry enough ...


dgsource("G:\Split.dgi")
changefps(last,last,true)

o = last  ox=o.width()  oy=o.height()

pre01  = o.sbr()
pre02  = pre01.medianblur(2,0,0)
pre03  = o.mt_makediff(mt_makediff(pre01,pre01.removegrain(4,-1)),U=2,V=2).sbr()

msk    = mt_lutxy(o.utoy(),o.vtoy(),"x 128 - 0 1 - * 128 + y + 2 / 148 - 16 *").greyscale()
brite  = pre02. levels(80,1.0,160,0,255,false).greyscale()
brite  = brite.bicubicresize(ox/2,oy/2)
brite2 = brite.repair(brite.repair(brite.mt_inpand().mt_inpand().mt_inpand().mt_inpand().mt_expand().mt_expand().mt_expand().mt_expand(),1,0),12,0)
bmask  = mt_lutxy(brite2.mt_inpand(),msk.mt_expand(),"x y -").mt_expand().removegrain(20).greyscale()
 \      .bicubicresize(ox,oy,1,0)

sss1   = mt_lutxy(pre02,o,"x x y - abs 1 1.22 / ^ 0.51 * x y - x y - abs 0.001 + / * -",U=2,V=2)
sss2   = sss1.temporalsoften(2,12,12,20,2).merge(pre02,0.14)
sss3   = sss1.mt_merge(sss2,bmask,luma=true,U=3,V=3)

sup0   = sss3.msuper(pel=pel1)
sup1   = o.msuper(pel=pel1,levels=1)
bv02   = sup0.manalyse(isb=true, delta=2,blksize=16,overlap=8 , search=5)#,searchparam=8,DCT=5)
bv01   = sup0.manalyse(isb=true, delta=1,blksize=16,overlap=8 , search=5)#,searchparam=8,DCT=5)
fv01   = sup0.manalyse(isb=false,delta=1,blksize=16,overlap=8 , search=5)#,searchparam=8,DCT=5)
fv02   = sup0.manalyse(isb=false,delta=2,blksize=16,overlap=8 , search=5)#,searchparam=8,DCT=5)

mdg1   = o.mdegrain2(sup1,bv01,fv01,bv02,fv02,thsad=400) 
mdg1a  = mdg1.mt_makediff(mt_makediff(mdg1,o).bicubicresize(ox/4,oy/4).bicubicresize(ox,oy,1,0),U=2,V=2)
mdg1aD = mt_makediff(o,mdg1)
pre02oD= mt_makediff(o,pre02)
xDD    = pre02oD.repair(mdg1aD,12)
spat2  = o.mt_makediff(xDD,U=2,V=2).minblur(0,2)
spat2  = mt_lutxy(spat2,o,"x 1 + y < x 2 + x 1 - y > x 2 - y ? ?",U=2,V=2)
spat2  = o.mt_merge(spat2,bmask,luma=true,U=3,V=3)

sup10  = mdg1.msuper(pel=pel2)
sup11  = spat2.FSx(mode=-1,sstr=2.0,cstr=-2.0,xstr=0.0).msuper(pel=pel2,sharp=1,levels=1)
bv4    = sup0.manalyse(isb=true, delta=4,blksize=16,overlap=8 , search=4,searchparam=1,DCT=5)
bv3    = sup0.manalyse(isb=true, delta=3,blksize=16,overlap=8 , search=5,searchparam=8,DCT=5)
bv2    = sup0.manalyse(isb=true, delta=2,blksize=16,overlap=8 , search=5,searchparam=8,DCT=5)
bv1    = sup0.manalyse(isb=true, delta=1,blksize=16,overlap=8 , search=5,searchparam=8,DCT=5)
fv1    = sup0.manalyse(isb=false,delta=1,blksize=16,overlap=8 , search=5,searchparam=8,DCT=5)
fv2    = sup0.manalyse(isb=false,delta=2,blksize=16,overlap=8 , search=5,searchparam=8,DCT=5)
fv3    = sup0.manalyse(isb=false,delta=3,blksize=16,overlap=8 , search=5,searchparam=8,DCT=5)
fv4    = sup0.manalyse(isb=false,delta=4,blksize=16,overlap=8 , search=4,searchparam=1,DCT=5)

         spat2.removegrain(11,0).mdegrain3(sup11,bv1,fv1,bv2,fv2,bv4,fv4,thsad=800)
mdg23  = last

         mt_makediff(mt_makediff(last,spat2).bicubicresize(ox/4,oy/4).bicubicresize(ox,oy,1,0),U=2,V=2)
         mt_lutxy(o,"x 4 + y < x 3 + x 4 - y > x 3 - x 0.49 * y 0.51 * + ? ?",U=3,V=3)
         last.mt_merge(mdg23,bmask,luma=true,U=3,V=3)
         mt_lutxy(sbr(),"x 4 + y < x 3 + x 4 - y > x 3 - x 0.51 * y 0.49 * + ? ?",U=2,V=2)

return(last)

# =========================================


# =========================================

# Helper functions

function sbr(clip o) 
{
rg11=o.removegrain(11)
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.removegrain(11)).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
o.mt_makediff(rg11DD,U=2,V=2) 
}

function MinBlur(clip clp, int r, int "uv")
{
uv   = default(uv,3)
uv2  = (uv==2) ? 1 : uv
rg4  = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==0) ? mt_makediff(clp,clp.sbr(),U=uv2,V=uv2)
 \    : (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
 \    : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
 \    :          mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D  = (r==1) ? mt_makediff(clp,clp.repair(clp.removegrain(4,rg4),14),U=uv2,V=uv2)
 \    : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
 \    :          mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD    = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}

function FSx(clip c, int "mode", float "sstr", float "cstr", float "xstr", float "lstr", float "pstr")
{
 
mode  = default(mode, 1   )   #  1 to 3, weakest to strongest. When negative -1 to -3, a broader kernel for equalisation is used.
sstr  = default(sstr, 2.0 )   #  strength of sharpening, 0.0 up to ??
 
_cstr = spline(sstr, 0,0, 0.5,0.1, 1.0,0.6, 2.0,0.9, 2.5,1.09, 3.0,1.15, 3.5,1.19, 4.0,1.249, 8.0,1.5, 255.0,2.0)

cstr  = default(cstr, _cstr)
xstr  = default(xstr, 0.19 )

lstr  = default(lstr, 4.0 )
pstr  = default(pstr, 1.6) 

str1  = sstr
str2  = cstr
SSTR  = string(abs(sstr))
CSTR  = string(abs(cstr))
LSTR  = string(abs(lstr))
PSTR  = string(abs(pstr))
rg=mode>0?11:20
 
b = (abs(mode)==1) ? c.removegrain(11,-1).removegrain(4,-1)
\ : (abs(mode)==2) ? c.removegrain(4,-1).removegrain(11,-1)
\ : (abs(mode)==3) ? c.removegrain(4,-1).removegrain(11,-1).removegrain(4,-1) : c
 
shrpD = mt_lutxy(c,b,"x y - abs "+LSTR+" / 1 "+PSTR+" / ^ "+LSTR+" * "+SSTR+" * x y - x y - abs 0.001 + / * x y - 2 ^ x y - 2 ^ "+SSTR+" 4.0 * + / * 128 +")
 
shrp = (str1<0.01) ? c : c.mt_adddiff(shrpD,U=2,V=2)

bzzD = (str2>0.0) ? shrpD.mt_lut("x 128 - "+CSTR+" * 128 +").removegrain(rg,-1)  
 \                : shrpD.mt_lut("x 128 - "+CSTR+" * 128 +").removegrain(11,-1).removegrain(20*1,-1)
shrp = (abs(str2)<0.01) ? shrp : shrp.mt_makediff(bzzD,U=2,V=2)

shrp = (xstr<0.01) ? shrp
\    : mt_lutxy(shrp,shrp.removegrain(20,-1),"x x y - 9.9 * +").repair(shrp,12,0).mergeluma(shrp,1.0-xstr)

return(shrp)
}

I think I like the result for the most part ... but at THAT speed, it would be out-of-question for me to use it. 1080p simply is not what you want to make any complicated MVTools-denoising with.

[Taurus]

@Didée:
I've looked at your second take.
Great improvement in the sky area and overall.
But......as you've mentioned, the script is slow as hell.
On my Core2Duo it would have take ages til the end
A resized version is acceptabel, so:
Thanks for sharing. It's something for my toolbox.

[Didée]

Originally Posted by Taurus

Great improvement in the sky area and overall.

Well ... especially when considering there's A BUG in the script - in the 2nd MDegrain instance two vectors are identical when they should be different. Copy+Paste mistake.
Also, I need to include one more helper - I've used a slightly modified FineSharp script. The given parameters are bad when used with the official version.

Correction to come, I'm about to start encoding the sample again.

edit - ah, with correct vectors, sample is 13570 instead of 14500 kbps.

But......as you've mentioned, the script is slow as hell.

Yes. However, I've used pretty slow MAnalyse settings for the 2nd instance. Perhaps one could ease those settings somewhat more ... I just wasn't keen on running another 20 test runs to see how much is needed or how low could be gone. I just aimed high with settings that are definetly sufficient, and let it be.

But anyway ... with 1080p, it's almost suicidal to use a 2nd instance of Manalyse/MDegrain that is referencing a 1st MDegrain instance as basis/source. Well, I can't help ... I fiddle together the sort of processing that seems capable to solve a problem. If it turns out the chain is too slow, ... then it is like it is. [*shrugs shoulders*]

[Taurus]

@Didée:
A small Typo at line 141 (:)

[phate89]

I tried the script on 5% of an episode.
I have to say, the result is great, a lot better than i expected. it loses a little bit of detail but it improves drastically the video quality and leaves a little bit of normal grain that helps to hide defects (if someone wants to look at it i have a crf 23 placebo to upload, crf 18 takes too much time at 40 kbps)
I understand very few of what exactly the script does but it works very well with my problem.
I think i'm gonna launch the first full episode now, and if i decide it's worth it i will start the full season.

If i can't manage to do it with your script i'll give a chance to 720p at least. It's all unother thing with a smaller resolution right? It's easier to clean well and preserve detail?

[phate89]

This week I finally find the time to look at the result of the rip of the full episode (it took me 5 days!). The result is very good but doing it for 13 episodes and probably for the next season is simply too much.. So I'm gonna do some quick try with 720p and if the visual result is not enough I'll keep the remux on the disk..
Is it easier in your opinion to get a good 720p or I still need something very complicated like the old script for 1080p?